IN-CONTEXT LEARNING

WinoDict: Probing language models for in-context word acquisition (2023.eacl-main)

Challenge:	Existing methods for data augmentation generate new examples wildly without proper control, which hinders the usefulness of the proposed approach.
Approach:	They propose a chain-of-thought attribute manipulation approach that generates new data from existing examples by tweaking in the user-provided attribute.
Outcome:	The proposed approach generates new data from existing examples by tweaking in the user-provided, task-specific attribute, e.g., sentiment polarity or topic in movie reviews.

Copied to clipboard

Julian Martin Eisenschlos, Jeremy R. Cole, Fangyu Liu, William W. Cohen

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation (2024.acl-long)

Challenge:	Large Language Models (LLMs) are unable to reflect the way language changes over time as their training corpus is frozen in time.
Approach:	They propose a new in-context learning paradigm to measure Large Language Models' ability to learn novel words during inference.
Outcome:	The proposed model improves on Winograd-style co-reference resolution problems by replacing the key concept word with a plausible word that the model must understand to complete the task.

Copied to clipboard

Shicheng Xu, Liang Pang, Mo Yu, Fandong Meng, Huawei Shen, Xueqi Cheng, Jie Zhou

Automated Clinical Data Extraction with Knowledge Conditioned LLMs (2025.coling-industry)

Challenge:	Existing studies show that LLMs face challenges in effectively using retrieved information . authors propose a method that considers LLM as "Information Refiner"
Approach:	They propose a method that considers LLMs as "Information Refiners" they propose INFO-RAG, which is low-cost and general across various tasks .
Outcome:	The proposed method improves performance of LLaMA2 by 9.39% relative points . it is low-cost and general across various tasks, and is robust and in-context learning is possible .

Copied to clipboard

Diya Li, Asim Kadav, Aijing Gao, Rui Li, Richard Bourgon

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes (2024.naacl-short)

Challenge:	Large language models (LLMs) can be effective at interpreting unstructured text in reports, but they often hallucinate due to a lack of domain-specific knowledge.
Approach:	They propose a framework that aligns generated internal knowledge with external knowledge through in-context learning (ICL) they use a retriever to identify relevant units of internal or external knowledge and a grader to evaluate the truthfulness and usefulness of the retrieved internal-knowledge rules to align and update the knowledge bases.
Outcome:	Experiments with expert-curated test datasets show that the proposed framework can increase the F1 score for key fields by 12.9% over existing methods.

Copied to clipboard

Harmon Bhasin, Timothy Ossowski, Yiqiao Zhong, Junjie Hu

PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from related Example Banks (2025.naacl-long)

Challenge:	Multi-task learning (MTL) for generalist models is a promising direction that offers transfer learning potential.
Approach:	They propose to combine multi-task learning (MTL) with in-context learning (ICL) to build models that can generalize to multiple tasks while being robust to out-of-distribution examples.
Outcome:	The proposed training strategies enable models to learn difficult tasks while mixing in prior tasks, denoted as mixed curriculum.

Copied to clipboard

Soumya Suvra Ghosal, Soumyabrata Pal, Koyel Mukherjee, Dinesh Manocha

Small Models are Valuable Plug-ins for Large Language Models (2024.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated impressive few-shot learning capabilities through in-context learning.
Approach:	They propose a novel Alternating Minimization approach for example selection that improves ICL performance on low-resource Indic languages.
Outcome:	The proposed approach outperforms existing frameworks for retrieving examples on low-resource Indic languages.

Copied to clipboard

Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, Julian McAuley

“Stupid robot, I want to speak to a human!” User Frustration Detection in Task-Oriented Dialog Systems (2025.coling-industry)

Challenge:	Large-scale pre-trained language models are difficult to fine-tune due to their huge weights and limited context length.
Approach:	They propose an approach which allows black-box LLMs to work with locally fine-tuned smaller models, resulting in superior performance on supervised tasks.
Outcome:	The proposed approach overcomes the challenges of poor performance and instability of In-Context Learning (ICL) while reducing the complexity of in-context learning.

Copied to clipboard

Mireia Hernandez Caralt, Ivan Sekulic, Filip Carevic, Nghia Khau, Diana Nicoleta Popa, Bruna Guedes, Victor Guimaraes, Zeyu Yang, Andre Manso, Meghana Reddy, Paolo Rosso, Roland Mathis

Sentiment Analysis

Challenge:	Detecting user frustration in task-oriented dialog systems is imperative for maintaining overall user satisfaction, engagement and retention.
Approach:	They compare out-of-the-box methods for user frustration detection with open-source methods . they find an LLM-based approach is promising, as it captures both emotion and dialog breakdowns .
Outcome:	The proposed method outperforms open-source methods in detecting user frustration in a TOD system.

Can LLM’s Generate Human-Like Wayfinding Instructions? Towards Platform-Agnostic Embodied Instruction Synthesis (2024.naacl-short)

Copied to clipboard

Vishnu Sashank Dorbala, Sanjoy Chowdhury, Dinesh Manocha

Challenge:	83.3% of users find the synthesized instructions accurately capture the details of the environment and show characteristics similar to those of human-generated instructions.
Approach:	They propose an algorithm that uses in-context learning to condition an LLM to generate instructions using just a few references.
Outcome:	The proposed algorithm is platform-agnostic and 83.3% of users find it to be accurate and similar to human-generated instructions.

MATSA: Multi-Agent Table Structure Attribution (2024.emnlp-demo)

Copied to clipboard

Puneet Mathur, Alexa Siu, Nedim Lipka, Tong Sun

Discourse-Aware In-Context Learning for Temporal Expression Normalization (2024.naacl-short)

Challenge:	Tabular data present unique challenges for attribution due to ambiguities, complex header hierarchies, and the difficulty in interpreting individual table cells without row and column context.
Approach:	They propose a task to generate row and column-level attributions supporting LLM-generated answers.
Outcome:	The proposed task outperforms baselines on tabCite and improves F1 score.

Copied to clipboard

Akash Gautam, Lukas Lange, Jannik Strötgen

Explore Spurious Correlations at the Concept Level in Language Models for Text Classification (2024.acl-long)

Challenge:	Temporal expression (TE) normalization is a well-studied problem, but upcoming machine learning approaches suffer from a lack of labeled data.
Approach:	They propose to use in-context learning to inject task, document, and example information into a large language model for temporal expression normalization.
Outcome:	The proposed model performs better in non-standard settings by dynamically including relevant examples during inference.

Copied to clipboard

Yuhang Zhou, Paiheng Xu, Xiaoyu Liu, Bang An, Wei Ai, Furong Huang

AMR-RE: Abstract Meaning Representations for Retrieval-Based In-Context Learning in Relation Extraction (2025.naacl-srw)

Challenge:	Language models have demonstrated remarkable performance in numerous NLP tasks, employing both fine-tuning and in-context learning (ICL) methods.
Approach:	They propose a method to assess concept bias in models during fine-tuning and in-context learning using ChatGPT.
Outcome:	The proposed method outperforms token removal approaches and is validated through extensive testing.

Copied to clipboard

Peitao Han, Lis Pereira, Fei Cheng, Wan Jou She, Eiji Aramaki

Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning (2023.emnlp-main)

Challenge:	Existing in-context learning methods for relation extraction often overlook entity relationships . Existing methods for RE prioritize language similarity over structural similarity .
Approach:	They propose an AMR-enhanced retrieval-based ICL method for relation extraction . their method retrieves in-context examples based on semantic structure similarity .
Outcome:	The proposed method outperforms baselines on four English RE datasets and in the more demanding unsupervised setting.

Copied to clipboard

Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, Jay Pujara

Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks (2025.findings-acl)

Challenge:	Temporal knowledge graphs (TKGs) are used to represent real-world facts in a structured way.
Approach:	They propose to use in-context learning with large language models for TKG forecasting . they compare naive LLMs to state-of-the-art (SOTA) supervised models .
Outcome:	The proposed approach performs well against pre-trained large language models . the proposed approach is based on simple heuristics and state-of-the-art models compared with pre-trainers .

Copied to clipboard

Yuntai Bao, Xuhong Zhang, Tianyu Du, Xinkui Zhao, Zhengwen Feng, Hao Peng, Jianwei Yin

Question Answering

Challenge:	Large language models (LLMs) are trained on vast corpora that contain substantial knowledge but their outputs often contain confidently stated inaccuracies.
Approach:	They propose to encode truthfulness as a distinct linear feature, termed the "truth direction", which can classify truthfulness reliably.
Outcome:	The proposed model can generalize to logical transformations, question-answering tasks, in-context learning, and external knowledge sources.

SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data (2023.findings-emnlp)

Copied to clipboard

Ruoxi Sun, Sercan Arik, Rajarishi Sinha, Hootan Nakhost, Hanjun Dai, Pengcheng Yin, Tomas Pfister

Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures (2026.findings-acl)

Challenge:	Text-to-SQL aims to automate the process of generating SQL queries on a database from natural language text.
Approach:	They propose a method to improve few-shot prompting capabilities of Text-to-SQL for Large Language Models (LLMs) they propose 'SQlPrompt' which aims to diversify the SQL proposals during consistency selection with different prompt designs and foundation models.
Outcome:	The proposed method outperforms previous approaches for in-context learning with zero labeled data by a large margin, closing the gap with finetuning state-of-the-art with thousands of labeles.

Copied to clipboard

Shenran Wang, Timothy Tin-Long Tse, Jian Zhu

Grounded Multimodal In-Context Learning for Product Weight Estimation at Scale in E-commerce (2026.acl-industry)

Challenge:	In-context learning is an emergent ability from pretrained Large Language Models (LLMs).
Approach:	They perform in-depth evaluations of in-context learning on transformers and hybrid large language models using behavioral probing and intervention-based methods.
Outcome:	The proposed model performs well on state-of-the-art transformer, state-space, and hybrid large language models.

Copied to clipboard

Bhavuk Singhal, Arsh Keshari, Ravindra Kumar Yadav

FLIRT: Feedback Loop In-context Red Teaming (2024.emnlp-main)

Challenge:	a large number of e-commerce platforms require manual verification and specialized hardware.
Approach:	They propose a multimodal weight estimation framework that uses category-specific exemplars to infer discretized weight buckets.
Outcome:	The proposed approach outperforms strong multimodal KNN baselines in accuracy and near-bucket reliability.

Copied to clipboard

Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta

MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models (2023.acl-short)

Challenge:	Recent work has evaluated the vulnerabilities of large generative models, such as DALL-E, ChatGPT, and GPT-4.
Approach:	They propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation.
Outcome:	The proposed framework evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation.

Copied to clipboard

Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin Yang, Kai-Wei Chang

Post-ASR Correction in Hindi: Comparing Language Models and Large Language Models in Low-Resource Scenarios (2026.eacl-short)

Challenge:	Large-scale pre-trained vision-language models do not possess the ability to conduct in-context learning.
Approach:	They propose to meta-train a language model to perform in-context learning on NLP tasks and then transfer this model to VL tasks by attaching a visual encoder.
Outcome:	The proposed model outperforms the baseline model on VQA, OK-VQA, and GQA while having 20 times fewer parameters.

Copied to clipboard

Rishabh Kumar, Amrith Krishna, Ganesh Ramakrishnan, Preethi Jyothi

PunchBench: Benchmarking MLLMs in Multimodal Punchline Comprehension (2025.acl-long)

Challenge:	Automatic Speech Recognition (ASR) systems for low-resource languages produce erroneous transcripts due to limited annotated data and linguistic complexity.
Approach:	They compare language models and large language models for post-ASR correction in Hindi . they observe a scaling trend under zero-shot ICL where mid-sized LLMs degrade performance before marginal recovery at extreme scales.
Outcome:	The proposed model outperforms larger models in both fine-tuning and in-context learning settings.

Copied to clipboard

Kun Ouyang, Yuanxin Liu, Shicheng Li, Yi Liu, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models (2024.naacl-short)

Challenge:	Existing benchmarks on punchline comprehension suffer from language shortcuts that allow models to rely on text, lack of question diversity, and narrow focus on a specific domain of multimodal content.
Approach:	They propose a multimodal punchline comprehension benchmark to assess models' ability to comprehend punchlines.
Outcome:	The proposed model surpasses in-context learning and chain-of-thought in punchline comprehension.

Copied to clipboard

Tingyu Xie, Qi Li, Yan Zhang, Zuozhu Liu, Hongwei Wang

Exploring Self-supervised Logic-enhanced Training for Large Language Models (2024.naacl-long)

Challenge:	Existing studies exploring the performance of large language models on named entity recognition tasks have focused on training task-specific LLMs for NER.
Approach:	They propose a training-free self-improving framework that utilizes an unlabeled corpus to stimulate the self-learning ability of LLMs.
Outcome:	The proposed framework improves performance on the named entity recognition task by using an unlabeled corpus.

Copied to clipboard

Fangkai Jiao, Zhiyang Teng, Bosheng Ding, Zhengyuan Liu, Nancy Chen, Shafiq Joty

Does GPT-3 Generate Empathetic Dialogues? A Novel In-Context Example Selection Method and Automatic Evaluation Metric for Empathetic Dialogue Generation (2022.coling-1)

Challenge:	Traditional attempts to enhance the logical reasoning abilities of language models often rely on supervised fine-tuning, limiting their generalization to new tasks or domains.
Approach:	They propose a framework for integrating logical reasoning capabilities into LLMs and activating them via in-context learning.
Outcome:	The proposed framework achieves comparable results to existing models on three language understanding benchmarks.

Copied to clipboard

Young-Jun Lee, Chae-Gyun Lim, Ho-Jin Choi

A Japanese News Simplification Corpus with Faithfulness (2024.lrec-main)

Challenge:	Empathy is a multi-dimensional concept consisting of cognitive and affective aspects.
Approach:	They propose two new in-context example selection methods that utilize emotion and situational information.
Outcome:	The proposed method is effective in measuring the degree of human empathy.

Copied to clipboard

Toru Urakawa, Yuya Taguchi, Takuro Niitsuma, Hideaki Tamori

SLENDER: Structured Outputs for SLM-based NER in Low-Resource Englishes (2025.acl-industry)

Challenge:	Existing simplified corpora lack faithfulness to original text, resulting in errors in translation.
Approach:	They propose to simplify Japanese newspaper articles to prioritize faithfulness over automated models.
Outcome:	The proposed corpus preserves the original text, surpassing existing corpora.

Copied to clipboard

Nicole Ren, James Teo

Language Models “Grok” to Copy (2025.naacl-short)

Challenge:	Named Entity Recognition (NER) for low-resource variants of English remains challenging, as most models are trained on datasets predominantly focused on American or British English.
Approach:	They propose a new output format for Named Entity Recognition (NER) that achieves a three-fold reduction in inference time compared to JSON format.
Outcome:	The proposed output format achieves a three-fold reduction in inference time on average compared to JSON format, which is widely used for structured outputs.

Copied to clipboard

Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Rui Yan

A Survey on In-context Learning (2024.emnlp-main)

Challenge:	We examine the pre-training dynamics of language models, focusing on their ability to copy text from preceding context.
Approach:	They propose that Transformer-based language models develop copying abilities similarly to grokking . they argue that the connection between groking and context copying can improve in-context performance.
Outcome:	The proposed model development is similar to grokking, but the speed is independent of tokens trained.

Copied to clipboard

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui

Self-Augmented In-Context Learning for Unsupervised Word Translation (2024.acl-short)

Challenge:	In-context learning (ICL) is a new paradigm for natural language processing . large language models (LLMs) demonstrate the ability to learn from a few examples .
Approach:	They propose to explore ICL to evaluate and extrapolate the ability of large language models.
Outcome:	The proposed methods can be used to evaluate and extrapolate the ability of large language models.

Copied to clipboard

Yaoyiran Li, Anna Korhonen, Ivan Vulić

In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation (2025.findings-naacl)

Challenge:	Large language models (LLMs) have strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, but they cannot match the performance of ‘traditional’ mapping-based approaches in the unsupervised scenario where no seed translation pairs are available.
Approach:	They propose a self-augmented in-context learning method that iteratively induces a set of high-confidence word translation pairs for in-constext learning from an LLM and reapplies them to the same LLM in the ICL fashion.
Outcome:	The proposed method shows substantial gains over zero-shot prompting of LLMs on two established benchmarks, outperforming mapping-based baselines across the board.

Copied to clipboard

Armel Randy Zebaze, Benoît Sagot, Rachel Bawden

Why Generate When You Can Discriminate? A Novel Technique for Text Classification using Language Models (2024.findings-eacl)

Challenge:	Existing studies have shown that in-context examples for machine translation are beneficial for high-resource languages.
Approach:	They propose to use in-context examples for machine translation (MT) they argue that similarity-based selection can improve MT .
Outcome:	The proposed approach improves machine translation (MT) and low-resource languages.

Copied to clipboard

Sachin Pawar, Nitin Ramrakhiyani, Anubhav Sinha, Manoj Apte, Girish Palshikar

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal (2024.acl-long)

Challenge:	Existing methods for text classification using autoregressive language models are limited . authors propose a novel technique for text classification using autoreregressives .
Approach:	They propose a two-step technique for text classification using autoregressive language models . they use a set of perplexity and log-likelihood based numeric features to elicit a text instance .
Outcome:	The proposed technique eliminates parameter updates in LMs and does not limit training examples . it is evaluated across 5 datasets and compares with multiple competent baselines .

Copied to clipboard

Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, Jinsong Su

Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding (2025.acl-short)

Challenge:	Existing methods to train LLMs on previous training data are not feasible in real-world applications because of catastrophic forgetting.
Approach:	They propose a framework that uses the LLM to generate synthetic instances for rehearsal and refine the instance outputs based on the synthetic inputs.
Outcome:	The proposed framework achieves superior or comparable performance compared to conventional rehearsal-based approaches while being more data-efficient.

Copied to clipboard

Keqin Peng, Liang Ding, Yuanxin Ouyang, Meng Fang, Yancheng Yuan, Dacheng Tao

Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering (2023.acl-long)

Challenge:	Prior research has found that large language models overlook input-label mapping information in ICL, relying more on their pre-trained knowledge.
Approach:	They propose a novel method that contrasts input-label mappings between positive and negative in-context examples to improve model performance.
Outcome:	The proposed method improves performance on 7 natural language understanding tasks without additional training.

Copied to clipboard

Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong

Emergent Abilities in Reduced-Scale Generative Language Models (2024.findings-naacl)

Challenge:	In-context learning is a common practice to randomly sample examples to serve as context.
Approach:	They propose a new principle for in-context learning that helps each sample find an in-constitut example organization that can derive the correct prediction.
Outcome:	The proposed method achieves 40% relative improvement over the common practice setting.

Copied to clipboard

Sherin Muckatira, Vijeta Deshpande, Vladislav Lialin, Anna Rumshisky

Transitive Consistency Constrained Learning for Entity-to-Entity Stance Detection (2024.acl-long)

Challenge:	Large language models can solve new tasks without task-specific fine-tuning.
Approach:	They propose to use pre-training data to pre-train 36 language models with billions of parameters to investigate whether emergent properties are tied to model size or can be demonstrated by smaller models.
Outcome:	The proposed model performs comparable to models trained on unrestricted language.

Copied to clipboard

Haoyang Wen, Eduard Hovy, Alexander Hauptmann

Sentiment Analysis

Challenge:	Entity-to-entity stance detection is a streamlined task without the complex dependency structure for structural sentiment analysis.
Approach:	They propose a method that models transitive consistency constraints during training to help train entity-to-entity stance detection models.
Outcome:	The proposed method improves both classification-based and generation-based models while large language models struggle with predicting link direction and neutral labels.

DRUM: Learning Demonstration Retriever for Large MUlti-modal Models (2025.acl-srw)

Copied to clipboard

Ellen Yi-Ge, Jiechao Gao, Wei Han, Wei Zhu

DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting (2025.findings-emnlp)

Challenge:	Recent studies have shown that large language models (LLMs) have impressive capabilities in dealing with new tasks with the help of in-context learning (ICL).
Approach:	They propose to concate the image and text embeddings to enhance the retrieval performance of a visual-language task and to calculate a list-wise ranking loss for training the embeddable model.
Outcome:	The proposed framework fine-tunes the CLIP embedding model to better meet the needs of the large vision-language models.

Copied to clipboard

Mingchen Li, Heng Fan, Song Fu, Junhua Ding, Yunhe Feng

Open-World Attribute Mining for E-Commerce Products with Multimodal Self-Correction Instruction Tuning (2025.acl-long)

Challenge:	Existing methods for prompt privacy focus on document-level rewriting, neglecting rich, multi-granular representations of text.
Approach:	a framework that leverages local differential privacy and composition theorem via group text rewriting is proposed . the framework is compatible with existing rewrite techniques and is publicly available at anonymous.4open.science for reproducibility.
Outcome:	DP-GTR is the first framework to integrate document-level and word-level information while exploiting in-context learning to improve privacy and utility.

Copied to clipboard

Jiaqi Li, Yanming Li, Xiaoli Shen, Chuanyi Zhang, Guilin Qi, Sheng Bi

Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? (2024.emnlp-main)

Challenge:	Current AM methods focus on extracting attributes from unimodal text, underutilizing multimodal data.
Approach:	They propose a framework for multimodal self-correction instruction tuning to extract new attributes from images and text with Multimodal Large Language Models.
Outcome:	The proposed framework outperforms state-of-the-art methods on two datasets.

Copied to clipboard

Zhe Yang, Yichang Zhang, Tianyu Liu, Jian Yang, Junyang Lin, Chang Zhou, Zhifang Sui

LIP-NER: Literal Patterns Benefit LLM-Based NER (2025.acl-srw)

Challenge:	Large language models (LLMs) have impressive capabilities, but still suffer from inconsistency issues.
Approach:	They develop a ConsisEval benchmark to evaluate LLMs' inconsistency . they find that LLM models can paradoxically fail at easier problems .
Outcome:	The proposed model achieves highest consistency score but inconsistent to specific questions due to distraction by redundant information, misinterpretation of questions, etc.

Copied to clipboard

Ruiqi Li, Li Chen

DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning (2025.naacl-long)

Challenge:	Existing methods for Named Entity Recognition (NER) use semantic information, but it is non-trivial to obtain literal patterns written in natural language.
Approach:	They propose an LLM-based NER framework that utilizes Literal Patterns to acquire literal patterns in natural language.
Outcome:	The proposed framework reduces human labor and provides a more efficient way to acquire literal patterns.

Copied to clipboard

Xinyu Tang, Xiaolei Wang, Xin Zhao, Ji-Rong Wen

Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction (2024.naacl-long)

Challenge:	Existing methods to conduct in-context learning without using human-annotated demonstrations are unreliable and lead to error accumulation.
Approach:	They propose a method to conduct in-context learning without using human-annotated demonstrations.
Outcome:	The proposed method outperforms existing methods using human-annotated demonstrations.

Copied to clipboard

Chenming Tang, Fanyi Qu, Yunfang Wu

Syntactic Text Processing

Challenge:	In-context learning (ICL) has shown impressive results on many tasks, but applying LLMs to grammatical error correction (GEC) is still a challenging task.
Approach:	They propose an ungrammatical-syntax-based in-context example selection strategy that measures similarity of sentences based on their syntactic structures and identify optimal ICL examples sharing the most similar ill-formed syntax to the test input.
Outcome:	The proposed strategy outperforms word-matching and semantics-based methods on a syntax-oriented task like GEC on benchmark English datasets.

BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer (2024.naacl-long)

Copied to clipboard

Akari Asai, Sneha Kudugunta, Xinyan Yu, Terra Blevins, Hila Gonen, Machel Reid, Yulia Tsvetkov, Sebastian Ruder, Hannaneh Hajishirzi

Instruction Induction: From Few Examples to Natural Language Task Descriptions (2023.acl-long)

Challenge:	Recent advances in few-shot generalization in natural language processing focus on English.
Approach:	They propose a benchmark that unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format and provides a fixed set of few-shot examples and instructions.
Outcome:	The proposed framework unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format and provides a fixed set of few-shot examples and instructions.

Copied to clipboard

Or Honovich, Uri Shaham, Samuel R. Bowman, Omer Levy

GrounDial: Human-norm Grounded Safe Dialog Response Generation (2024.findings-eacl)

Challenge:	Large language models can perform unseen tasks by conditioning on a few input-output demonstrations, but task inference is implicit and the ability of models to explicitly reason about it remains unexplored.
Approach:	They propose an instruction induction challenge in which a model is asked to generate a natural language instruction that fits a set of labeled examples.
Outcome:	The proposed model achieves 65.7% of human performance while the original model only reaches 9.8% of human performances.

Copied to clipboard

Siwon Kim, Shuyang Dai, Mohammad Kachuee, Shayan Ray, Tara Taghavi, Sungroh Yoon

Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations (2025.findings-naacl)

Challenge:	Recent conversational AI systems generate unsafe responses agreeing to offensive user input or including toxic content.
Approach:	They propose a method where response safety is achieved by grounding responses to commonsense social rules without fine-tuning.
Outcome:	The proposed approach is quantitatively and qualitatively safer even without additional data or tuning.

Copied to clipboard

Wenjie Jacky Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Hadi Askari, Chaowei Xiao, Muhao Chen

KwaiChat: A Large-Scale Video-Driven Multilingual Mixed-Type Dialogue Corpus (2025.findings-naacl)

Challenge:	Existing studies on backdoor defense have focused on training phase, overlooking critical aspect of testing time defense.
Approach:	They propose to use demonstrations as a defense mechanism against backdoor attacks in black-box LLMs.
Outcome:	The proposed method outperforms existing defense baselines across most evaluation scenarios.

Copied to clipboard

Xiaoming Shi, Zeming Liu, Yiming Lei, Chenkai Zhang, Haitao Leng, Chuan Wang, Qingjie Liu, Wanxiang Che, Yunhong Wang

Natural Language Interfaces

Challenge:	Currently, video-based dialogue systems rely on a single dialogue type, hindering their versatility in practical applications.
Approach:	They propose to generate video-driven multilingual mixed-type dialogues using KwaiChat . they propose to create a video-based multilingual mix of 4 dialogue types, 30 domains, 4 languages, 13 topics .
Outcome:	The proposed model performs best on KwaiChat but is not perfect in this situation.

XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages (2023.findings-emnlp)

Copied to clipboard

Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelani, Vera Axelrod, Isaac Caswell, Colin Cherry, Dan Garrette, Reeve Ingle, Melvin Johnson, Dmitry Panteleev, Partha Talukdar

Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations (2023.acl-long)

Challenge:	Existing datasets are often informed by established research directions in the NLP community.
Approach:	They propose a benchmark to evaluate the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks.
Outcome:	The proposed benchmark evaluates the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks.

Copied to clipboard

Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi

MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretations (2023.emnlp-main)

Challenge:	Existing methods for zero-shot learning are based on in-context training, but performance drops when no demonstrations are available.
Approach:	They propose a new method that constructs pseudo-demonstrations for a given test input using a raw text corpus and applies techniques to reduce copying.
Outcome:	The proposed method outperforms previous zero-shot methods on nine classification datasets and is on par with in-context learning with labeled training data in the few-shot setting.

Copied to clipboard

Arkil Patel, Satwik Bhattamishra, Siva Reddy, Dzmitry Bahdanau

ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classification (2024.findings-naacl)

Challenge:	Large Language Models (LLMs) have a knowledge cutoff and are costly to finetune repeatedly.
Approach:	They introduce a language evaluation suite that incorporates diverse tokens and prompt settings to simulate real-world complexity.
Outcome:	The proposed evaluation suite incorporates diverse tokens and prompt settings to simulate real-world complexity.

Copied to clipboard

Yaxin Zhu, Hamed Zamani

uMedSum: A Unified Framework for Clinical Abstractive Summarization (2025.acl-long)

Challenge:	Existing research has focused on fully supervised XMC, but real-world scenarios often lack supervision signals, highlighting the importance of zero-shot settings.
Approach:	They propose a framework that generates a set of candidate labels through in-context learning and then reranks them.
Outcome:	The proposed framework advances state-of-the-art on two diverse public benchmarks.

Copied to clipboard

Aishik Nagar, Yutong Liu, Andy T. Liu, Viktor Schlegel, Vijay Prakash Dwivedi, Arun-Kumar Kaliya-Perumal, Guna Pratheep Kalanchiam, Yili Tang, Robby T. Tan

TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Data (2025.acl-long)

Challenge:	Clinical abstractive summarization struggles to balance faithfulness and informativeness, sacrificing key information or introducing confabulations.
Approach:	They develop a modular hybrid framework that integrates confabulation removal and key information addition into abstractive summarization methods.
Outcome:	The proposed framework outperforms state-of-the-art abstractive summarization methods in both quantitative metrics and expert evaluations.

Copied to clipboard

Xiang Huang, Jiayu Shen, Shanshan Huang, Sitao Cheng, Xiaxia Wang, Yuzhong Qu

Attack Prompt Generation for Red Teaming and Defending Large Language Models (2023.findings-emnlp)

Challenge:	Existing methods for semantic parsing rely on extensive manually annotated datasets and limited generalization capability to unseen examples.
Approach:	They propose a framework that generates high-relevance synthetic data without manual annotation . they generate queries for the queries and use them as demonstrations for in-context learning .
Outcome:	The proposed framework outperforms non-fine-tuned methods on KBQA datasets and shows superior sample efficiency, robustness, and generalization capabilities under non-I.I.D. settings.

Copied to clipboard

Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, Xiangnan He

DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework (2023.findings-emnlp)

Challenge:	Existing studies construct attack prompts via manual or automatic methods, but these methods have limitations on cost and quality.
Approach:	They propose an attack framework to instruct LLMs to mimic human-generated prompts through in-context learning and a defense framework that fine-tunes victim LLM's through iterative interactions with the attack framework.
Outcome:	The proposed approach is based on experiments on different LLMs to evaluate their effectiveness against red teaming attacks.

Copied to clipboard

Keonwoo Kim, Younggun Lee

Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction (2023.findings-emnlp)

Challenge:	Existing methods for few-shot topic classification are limited due to the volume of information pouring in from the Internet . a new framework is proposed to train a classifier for few shot topics .
Approach:	They propose a framework to train a classifier for few-shot topic classification using a customized dataset and a dense retriever model.
Outcome:	The proposed framework shows superior performance on few-shot topic classification tasks compared to baselines that use in-context learning .

Copied to clipboard

Xilai Ma, Jing Li, Min Zhang

Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations (2022.emnlp-main)

Challenge:	Existing approaches to few-shot relation extraction require training.
Approach:	They propose a method for few-shot relation extraction using large language models, called CoT-ER, chain-of-thought with explicit evidence reasoning.
Outcome:	The proposed approach achieves competitive performance compared to the fully-supervised state-of-the-art approach on the FewRel1.0 and FewRela2.0 datasets.

Copied to clipboard

Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang-goo Lee, Taeuk Kim

Meta-Learning of Prompt Generation for Lightweight Prompt Engineering on Language-Model-as-a-Service (2023.findings-emnlp)

Challenge:	Intuitively, ground-truth labels should have as much impact in in-context learning as supervised learning, but the impact of the quality of demonstrations remains elusive.
Approach:	They propose to measure input-label correspondence and ground-truth label effect ratio . they propose to use verbosity of prompt templates and language model size as controlling factors .
Outcome:	The proposed metrics show that ground-truth labels have less impact than previously thought . the authors identify key components as controlling factors to achieve noise-resilient ICL .

Copied to clipboard

Hyeonmin Ha, Jihye Lee, Wookje Han, Byung-Gon Chun

Is This LLM Library Learning? Evaluation Must Account For Compute and Behaviour (2026.eacl-long)

Challenge:	Language-Model-as-a-Services (LMaaSs) support a variety of user tasks through in-context learning from prompts.
Approach:	They propose a lightweight automatic prompt generation method that meta-trains a prompt generation model to enable robust learning from the contexts created by the generated prompts.
Outcome:	The proposed method improves performance on unseen tasks by 19.4% compared to the state-of-the-art prompt generation method.

Copied to clipboard

Ian Berlot-Attwell, Tobias Sesterhenn, Frank Rudzicz, Xujie Si

Small Language Models Improve Giants by Rewriting Their Outputs (2024.eacl-long)

Challenge:	Recent advances in the coding, reasoning, and tool-use ability of LLMs have raised the possibility of library learning with LLM.
Approach:	They propose to use reusable and composable functions and tools to create reusable, composesable code and tools that can be reused by modifying relevant examples.
Outcome:	The proposed system fails to consistently outperform the baseline model and does not correct for the difference in computational cost.

Copied to clipboard

Giorgos Vernikos, Arthur Brazinskas, Jakub Adamek, Jonathan Mallinson, Aliaksei Severyn, Eric Malmi

AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation (2026.eacl-long)

Challenge:	despite impressive performance of large language models, they lag behind specialized models in various tasks.
Approach:	They propose a training model that can be integrated with different LLMs at inference to improve their performance without task-specific training.
Outcome:	The proposed model outperforms standard models on four natural language generation tasks.

Copied to clipboard

Potsawee Manakul, Woody Haosheng Gan, Michael J Ryan, Ali Sartaz Khan, Warit Sirichotedumrong, Kunat Pipatanakul, William Barr Held, Diyi Yang

What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception (2024.naacl-long)

Challenge:	Current speech evaluation systems rely on specialized systems for individual audio characteristics and poor correlation between automatic methods and human preferences.
Approach:	They propose a unified evaluation framework for Large Audio Models as a Judge, AudioJudge . they propose specialized judges that can be prompted to perform audio characteristic detection tasks .
Outcome:	The proposed method improves performance across audio characteristic detection and human preference simulation tasks.

Copied to clipboard

Chaitanya Malaviya, Subin Lee, Dan Roth, Mark Yatskar

Extracting and Understanding the Superficial Knowledge in Alignment (2025.naacl-long)

Challenge:	Question answering models can often be black boxes, as their reasoning process is mostly opaque.
Approach:	They analyze the effect of rationales generated by QA models on user feedback and how well they enable users to understand and trust model answers.
Outcome:	The proposed model can be used to improve model responses by removing feedback from end users and enhancing model outputs by using natural language feedback.

Copied to clipboard

Runjin Chen, Gabriel Jacob Perin, Xuxi Chen, Xilun Chen, Yan Han, Nina S. T. Hirata, Junyuan Hong, Bhavya Kailkhura

Challenge:	Recent studies have shown that alignment of large language models with human values and preferences requires substantial data and computation resources.
Approach:	They propose a method to extract and isolate superficial knowledge from aligned models by focusing on the shallow modifications to the final token selection process.
Outcome:	The proposed method extracts and isolates superficial knowledge from aligned models, focusing on the shallow modifications to the final token selection process.

From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning (2025.naacl-long)

Copied to clipboard

Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

Scaling Sentence Embeddings with Large Language Models (2024.findings-emnlp)

Challenge:	Motivated by in-context learning capabilities of Large Language Models (LLMs), multimodal LLMs with additional visual modality are also exhibited with similar ICL abilities when multiple image-text pairs are provided as demonstrations.
Approach:	They conduct systematic and principled evaluation of multimodal ICL for models of different scales on a broad spectrum of new yet critical tasks.
Outcome:	The proposed model performance improves on a broad spectrum of new yet critical tasks.

Copied to clipboard

Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, Fuzhen Zhuang

From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning (2025.findings-emnlp)

Challenge:	Current methods based on contrastive learning have generated high-quality sentence embeddings.
Approach:	They propose a method to enhance LLM performance on sentence embeddings with a one-word limitation.
Outcome:	The proposed method outperforms contrastive learning methods on sentence embeddings without fine-tuning and with fine-untun.

Copied to clipboard

Zihan Chen, Song Wang, Xingbo Fu, Chengshuai Shi, Zhenyu Lei, Cong Shen, Jundong Li

Uncertainty Quantification for In-Context Learning of Large Language Models (2024.naacl-long)

Challenge:	In-context learning (ICL) enables large language models to perform novel tasks without parameter updates by conditioning on a few input-output examples.
Approach:	They propose a cost-efficient two-stage pipeline that reduces reliance on LLMs for data labeling.
Outcome:	The proposed pipeline reduces reliance on LLMs for data labeling . it leverages readily available cross-task examples to prompt an LLM and pseudo-label a small set of target task instances.

Copied to clipboard

Chen Ling, Xujiang Zhao, Xuchao Zhang, Wei Cheng, Yanchi Liu, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Jie Ji, Guangji Bai, Liang Zhao, Haifeng Chen

Timeline-based Sentence Decomposition with In Context Learning for Temporal Fact Extraction (2024.acl-long)

Challenge:	Existing studies on in-context learning have focused on quantifying the uncertainty associated with the model's response, but they neglect the complexity of the LLM and the uniqueness of in-constitut learning.
Approach:	They propose a method to quantify the uncertainty associated with in-context learning and propose corresponding estimation method to quantify both types of uncertainties.
Outcome:	The proposed method offers an unsupervised way to understand the prediction of in-context learning in a plug-and-play fashion.

Copied to clipboard

Jianhao Chen, Haoyuan Ouyang, Junyang Ren, Wentao Ding, Wei Hu, Yuzhong Qu

Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison (2024.findings-naacl)

Challenge:	Recent research on temporal fact extraction fails to establish time-to-fact correspondences in complex sentences.
Approach:	They propose a timeline-based sentence decomposition strategy using large language models with in-context learning to extract temporal facts from natural language text.
Outcome:	The proposed method achieves state-of-the-art on a complex temporal fact extraction dataset.

Copied to clipboard

Maxime Bouthors, Josep Crego, François Yvon

Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER (2022.acl-long)

Challenge:	Existing approaches to extract examples from memory are limited, but the upstream retrieval step is still unexplored.
Approach:	They propose to use a standard autoregressive model, edit-based model and a large language model with in-context learning to investigate the effect of retrieval methods on translation scores.
Outcome:	The proposed architectures improve translation scores and increase diversity of examples.

Copied to clipboard

Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren

Interpret and Improve In-Context Learning via the Lens of Input-Label Mappings (2025.acl-long)

Challenge:	Recent advances in prompt-based learning have shown strong results on few-shot text classification by using cloze-style templates.
Approach:	They propose a demonstration-based learning method which lets the input be prefaced by task demonstrations for in-context learning.
Outcome:	The proposed method improves on in-domain learning and domain adaptation in low-resource settings.

Copied to clipboard

Chenghao Sun, Zhen Huang, Yonggang Zhang, Le Lu, Houqiang Li, Xinmei Tian, Xu Shen, Jieping Ye

Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts (2022.findings-emnlp)

Challenge:	Large language models excel at downstream NLP tasks through in-context learning . however, the internal mechanisms behind ICL remain under-explored .
Approach:	They propose a PC patching approach to identify modules where input-label mappings function . they observe and verify that key heads utilize input-labeled mappings to generate target labels for new queries.
Outcome:	The proposed approach detects modules where input-label mappings function . it also detects that key heads use the mappings to generate labels for new queries .

Copied to clipboard

Nghia T. Le, Fan Bai, Alan Ritter

Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications (2024.naacl-long)

Challenge:	In-context learning has emerged as a promising approach to resolve anaphora, but there are challenges in applying it to scientific protocols.
Approach:	They propose a method which combines predictions of hundreds of in-context experts and combines them to yield a 30% increase in F1 over a competitive prompt retrieval baseline.
Outcome:	The proposed method yields 30% increase in F1 score over a competitive prompt retrieval baseline.

Copied to clipboard

Yanchen Liu, Srishti Gautam, Jiaqi Ma, Himabindu Lakkaraju

Text Classification

Challenge:	Recent studies suggest using large language models to make tabular classifications . however, LLMs have been shown to exhibit harmful social biases based on stereotypes and inequalities present in society.
Approach:	They propose to use large language models to make tabular classifications . they show that LLMs inherit biases from their training data .
Outcome:	The proposed models exhibit harmful biases that reflect stereotypes and inequalities in society.

Unsupervised Human Preference Learning (2024.emnlp-main)

Copied to clipboard

Sumuk Shashidhar, Abhinav Chinta, Vaibhav Sahai, Dilek Tur

MindAgent: Emergent Gaming Interaction (2024.findings-naacl)

Challenge:	Existing methods for in-context learning and parameter-efficient fine-tuning fail to capture the complexity of human preferences, especially given the small, personal datasets individuals possess.
Approach:	They propose a method that uses small parameter models as preference agents to generate natural language rules that guide a larger, pre-trained model, enabling efficient personalization.
Outcome:	The proposed method outperforms baseline personalization methods on email and article datasets and significantly outperformed existing methods.

Copied to clipboard

Ran Gong, Qiuyuan Huang, Xiaojian Ma, Yusuke Noda, Zane Durante, Zilong Zheng, Demetri Terzopoulos, Li Fei-Fei, Jianfeng Gao, Hoi Vo

MetaICL: Learning to Learn In Context (2022.naacl-main)

Challenge:	Large foundation models (LFMs) can perform complex scheduling in a multi-agent system and can coordinate agents to complete complex tasks that require extensive collaboration.
Approach:	They propose a gaming-based infrastructure that evaluates LFMs' planning and coordination capabilities in the context of gaming interaction.
Outcome:	The proposed infrastructure can be deployed in a customized VR version of Cuisineworld and adapted in the “Minecraft” domain.

Copied to clipboard

Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi

Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs (2025.acl-long)

Challenge:	Large language models can do in-context learning by conditioning on a few training examples with no parameter updates or task-specific templates.
Approach:	They propose a meta-training framework where a pretrained language model is tuned to do in-context learning on a large set of training tasks.
Outcome:	The proposed framework outperforms baseline models on 142 NLP datasets and a range of target tasks with domain shifts.

Copied to clipboard

Zhe Yang, Yichang Zhang, Yudong Wang, Ziyao Xu, Junyang Lin, Zhifang Sui

Challenge:	Existing approaches to improve self-correction performance of Large Language Models are based on intrinsic selfcorrectione, which allows the model to check and revise its selfgenerated answers without external feedback.
Approach:	They propose to decompose the self-correction capability into confidence and critique capabilities and a metric for overall self-corretion capability evaluation.
Outcome:	The proposed method outperforms vanilla SFT and achieves much higher accuracy after self-correction.

LaRS: Latent Reasoning Skills for Chain-of-Thought Reasoning (2024.findings-emnlp)

Copied to clipboard

Zifan Xu, Haozhu Wang, Dmitriy Bespalov, Xian Wu, Peter Stone, Yanjun Qi

Text-centric Alignment for Bridging Test-time Unseen Modality (2025.findings-emnlp)

Challenge:	Existing methods require human experts or pre-trained LLMs to describe the skill to guide the selection.
Approach:	They propose a new approach that uses unsupervised learning to create a latent space representation of rationales with a variable called a reasoning skill.
Outcome:	Empirical results show that LaRS outperforms SOTA skill-based selection methods . it processes example banks four times faster and reduces LLM inferences by half .

Copied to clipboard

Yun-Da Tsai, Ting-Yu Yen, Pei-Fu Guo, Zhe-Yan Li, Shou-De Lin

Challenge:	a text-centric alignment method is used to handle unseen modalities and dynamic modality combinations at test time.
Approach:	They propose a text-centric alignment method that unifies different input modalities into a single semantic text representation by leveraging in-context learning with Large Language Models and uni-modal foundation models.
Outcome:	The proposed method unifies input modalities into a single semantic representation . it significantly improves the ability to manage unseen, diverse, and unpredictable modality combinations .

Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning (2024.findings-naacl)

Copied to clipboard

Jianing Wang, Chengyu Wang, Chuanqi Tan, Jun Huang, Ming Gao

Extractive Summarization via ChatGPT for Faithful Summary Generation (2023.findings-emnlp)

Challenge:	Existing studies have explored multiple aspects that affect the performance of large language models (LLMs) such as input-output mapping, extensive data resources, and the ability to train on labeled examples.
Approach:	They propose a framework that injects knowledge into LLMs during continual self-supervised pre-training and judiciously selects examples with high knowledge relevance.
Outcome:	The proposed framework outperforms baseline models and improves by more than 13% and 7% on text classification and question-answering tasks.

Copied to clipboard

Haopeng Zhang, Xiao Liu, Jiawei Zhang

GPT-RE: In-context Learning for Relation Extraction using Large Language Models (2023.emnlp-main)

Challenge:	Abstractive summarization methods struggle with generating ungrammatical or even nonfactual contents.
Approach:	They evaluate ChatGPT's performance on extractive summarization and compare it with traditional fine-tuning methods on benchmark datasets.
Outcome:	The proposed pipeline performs better than abstractive methods on summary faithfulness and in-context learning.

Copied to clipboard

Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, Sadao Kurohashi

Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment (2023.emnlp-main)

Challenge:	Existing approaches to in-context learning (ICL) are lacking in relation extraction (RE) . emergence of large language models (LLMs) such as GPT-3 represents a significant advancement in natural language processing.
Approach:	They propose to incorporate task-aware representations into demonstration retrieval and enrich the demonstrations with gold label-induced reasoning logic.
Outcome:	The proposed model achieves SOTA and competitive performances on the Semeval and SciERC datasets.

Copied to clipboard

Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, Smaranda Muresan

Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization (2026.acl-long)

Challenge:	Current research on developing computational models of social norms has focused on American society.
Approach:	They propose to leverage a Chinese Q&A platform and a socialchiemistry dataset as proxies for contrasting cultural axes and align social situations cross-culturally.
Outcome:	The proposed model can reason across cultures using a Chinese Q&A platform and the existing socialChemistry dataset.

Copied to clipboard

Xu Chu, Guanyu Wang, Zhijie Tan, Xinrong Chen, Ziyu Li, Tong Mo, Weiping Li

Interpretability-based Tailored Knowledge Editing in Transformers (2024.emnlp-main)

Challenge:	Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model’s inherent order bias unresolved.
Approach:	They propose Dual Group Advantage Optimization (DGAO) which aims to improve model accuracy and order stability simultaneously.
Outcome:	The proposed method improves model accuracy and order stability while penalizing order-sensitive or incorrect responses.

Copied to clipboard

Yihuai Hong, Aldo Lipani

In-context Mixing (ICM): Code-mixed Prompts for Multilingual LLMs (2024.acl-long)

Challenge:	Existing methods for modifying in-context learning fail to analyze the instability of in-constitu learning outcomes.
Approach:	They propose a model-based knowledge editing method that considers the unique information flow of each sample and aims to correct errors without costly retraining.
Outcome:	The proposed method exploits the critical role of feed-forward MLPs in decoder-only models and reveals diverse attribute recall across transformer layers, guiding edits to specific features at different depths and mitigating over-editing issues.

Copied to clipboard

Bhavani Shankar, Preethi Jyothi, Pushpak Bhattacharyya

Deep Natural Language Feature Learning for Interpretable Prediction (2023.emnlp-main)

Challenge:	In-context mixing is a prompting technique for effective in-contact learning with multilingual large language models.
Approach:	They propose a prompting technique called in-context mixing for effective in-constext learning with multilingual large language models.
Outcome:	The proposed prompts perform better with multilingual large language models.

Copied to clipboard

Felipe Urrutia, Cristian Calderon, Valentin Barriere

Improving In-Context Learning with Prediction Feedback for Sentiment Analysis (2024.findings-acl)

Challenge:	Using a small transformer language model, we can break down a complex task into a set of intermediary easier sub-tasks.
Approach:	They propose a method to break down a main task into a set of intermediary easier sub-tasks, which are formulated in natural language as binary questions related to the final target task.
Outcome:	The proposed method breaks down a complex task into a set of easier sub-tasks, which are formulated in natural language as binary questions related to the final target task.

Copied to clipboard

Hongling Xu, Qianlong Wang, Yice Zhang, Min Yang, Xi Zeng, Bing Qin, Ruifeng Xu

Adapting Language Models to Compress Contexts (2023.emnlp-main)

Challenge:	Large language models (LLMs) have achieved promising results in sentiment analysis through the in-context learning paradigm.
Approach:	They propose a framework that incorporates prior predictions and feedback to improve sentiment understanding by incorporating prior feedback and leveraging a feedback-driven prompt.
Outcome:	The proposed framework improves on nine sentiment analysis datasets with an average improvement of 5.95% over conventional methods.

Copied to clipboard

Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen

WebWISE: Unlocking Web Interface Control for LLMs via Sequential Exploration (2024.findings-naacl)

Challenge:	Transformer-based language models have a finite context window and expensive computational cost of processing long text documents.
Approach:	They propose to adapt pre-trained LMs into AutoCompressors to compress text into summary vectors . authors propose to use summary vector to speed up inference over long contexts based on a finite context window .
Outcome:	The proposed model can compress long contexts into summary vectors, which are accessible as soft prompts.

Copied to clipboard

Heyi Tao, Sethuraman T V, Michal Shlapentokh-Rothman, Tanmay Gupta, Heng Ji, Derek Hoiem

Effective Self-Mining of In-Context Examples for Unsupervised Machine Translation with LLMs (2025.findings-naacl)

Challenge:	Prior work to control software has used reinforcement learning (RL), requiring many demonstrations and trials to learn simple interaction tasks.
Approach:	They propose a Large Language Model to automatically perform web software tasks using click, scroll, and text in- put operations using filtered Document Object Models as observations.
Outcome:	The proposed method performs better on the MiniWob++ benchmark with only one in-context example.

Copied to clipboard

Abdellah El Mekki, Muhammad Abdul-Mageed

A Survey on Training-free Alignment of Large Language Models (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated impressive performance on a wide range of natural language processing tasks.
Approach:	They propose an unsupervised approach to mine in-context examples for machine translation (MT) they use word-level mining to acquire word translations that are then used to perform sentence-level mines .
Outcome:	The proposed approach outperforms state-of-the-art methods on 288 directions on 287 languages and is based on word-level mining and sentence-level extraction.

Copied to clipboard

Birong Pan, Yongqi Li, Weiyu Zhang, Wenpeng Lu, Mayi Xu, Shen Zhou, Yuanyuan Zhu, Ming Zhong, Tieyun Qian

Rectifying Demonstration Shortcut in In-Context Learning (2024.naacl-long)

Challenge:	a survey of large language models (LLMs) aims to ensure outputs adhere to human values, ethical standards, and legal norms.
Approach:	They present the first systematic review of TF alignment methods . they categorize them by stages of pre-decoding, in-decoder and post-decoration .
Outcome:	The proposed methods are based on training-free (TF) alignment techniques . they are able to be used in open-source and closed-source environments without retraining .

Copied to clipboard

Joonwon Jang, Sanghwan Jang, Wonbin Kweon, Minjin Jeon, Hwanjo Yu

Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models (2024.findings-naacl)

Challenge:	Large language models (LLMs) can solve tasks with a few demonstrations, but often rely on their pre-trained semantic priors rather than the input-label relationships to proceed with ICL prediction.
Approach:	They propose a demonstration-aware calibration method to improve LLMs' ability to learn new input-label relationships from demonstrations.
Outcome:	The proposed method improves the original ICL task and the task learning setting, and the results are generalized across three LLM families.

Copied to clipboard

Wei He, Shichun Liu, Jun Zhao, Yiwen Ding, Yi Lu, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World (2025.findings-emnlp)

Challenge:	Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries.
Approach:	They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation.
Outcome:	The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks.

Copied to clipboard

Saeed Almheiri, Rania Elbadry, Mena Attia, Chenxi Wang, Preslav Nakov, Timothy Baldwin, Fajri Koto

CliniBench: A Clinical Outcome Prediction Benchmark for Generative and Encoder-Based Language Models (2026.eacl-long)

Challenge:	a recent study examined the potential for cross-cultural transfer of commonsense reasoning . merely 12 culture-specific examples from one country can improve performance in others by 10% on average .
Approach:	They evaluate cross-cultural transfer of commonsense reasoning within the arab world . they use in-context learning and demonstration-based reinforcement to evaluate alignment methods .
Outcome:	The proposed model can improve performance in cultures with cultural similarities in the Arab world by 10% on average.

Copied to clipboard

Paul Grundmann, Jan Frick, Dennis Fast, Thomas Steffek, Felix Gers, Wolfgang Nejdl, Alexander Löser

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)

Challenge:	generative large language models are being investigated for complex medical tasks, but their effectiveness in real-world clinical applications remains underexplored.
Approach:	They propose to compare encoder-based classifiers and generative LLMs for discharge diagnosis prediction from admission notes in a MIMIC-IV dataset.
Outcome:	The proposed benchmark compares encoder-based classifiers and generative LLMs for discharge diagnosis prediction from admission notes in the MIMIC-IV dataset.

Copied to clipboard

Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei

Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels (2025.coling-main)

Challenge:	Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear .
Approach:	They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent .
Outcome:	The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update .

Copied to clipboard

Chaoqun Liu, Qin Chao, Wenxuan Zhang, Xiaobao Wu, Boyang Li, Anh Tuan Luu, Lidong Bing

Par-ITA: Benchmarking Seq2Seq and LLMs on a Human-Supervised Parallel Corpus for Italian Hyperpartisan Neutralization (2026.acl-long)

Challenge:	Pre-trained language models have demonstrated remarkable performance through supervised fine-tuning or in-context learning using gold labels.
Approach:	They propose a new paradigm termed zero-to-strong generalization that prompts LLMs to annotate unlabeled data and retain high-quality labels by filtering.
Outcome:	The proposed framework outperforms pre-trained language models on extensive classification and reasoning tasks on multiple model sizes.

Copied to clipboard

Michele Joshua Maggini, Søren Fomsgaard, Michele Maestroni, Gaël Dias, Pablo Gamallo

ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context (2024.findings-emnlp)

Challenge:	a new study examines the role of hyperpartisan content in online polarization in the social web.
Approach:	They propose a human-supervised parallel corpus for italian hyperpartisan neutralization of 2,475 paragraph pairs.
Outcome:	The proposed dataset is the first human-supervised parallel corpus for italian hyperpartisan neutralization of 2,475 paragraph pairs.

Copied to clipboard

Zirui Wu, Yansong Feng

Unified Demonstration Retriever for In-Context Learning (2023.acl-long)

Challenge:	Tables are a crucial tool for organizing and presenting information in various domains.
Approach:	They propose a Plan-then-Reason framework to answer different types of user queries over tables with sentence context.
Outcome:	The proposed framework outperforms existing frameworks without self-consistency while using less API calls and in-context demonstrations.

Copied to clipboard

Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu

Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: A Case Study on Hateful Memes (2024.lrec-main)

Challenge:	In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction.
Approach:	They propose a single model to retrieve demonstrations for a wide range of tasks by combining training signals from various tasks into a unified list-wise ranking formulation by language model’s feedback.
Outcome:	The proposed model outperforms baselines on 30+ tasks across 13 task families and multiple data domains.

Copied to clipboard

Yosuke Miyanishi, Minh Le Nguyen

2INER: Instructive and In-Context Learning on Few-Shot Named Entity Recognition (2023.findings-emnlp)

Challenge:	Causal analyses define semantics, while gradient-based methods are essential to eXplainable AI (XAI), interpreting the model’s ‘black box’.
Approach:	They propose to integrate causal analysis and XAI to integrate a model's mechanisms into their analysis by integrating a dataset of hateful meme detection models.
Outcome:	The proposed model can detect hateful memes using intersectionality principles and summarized attention scores highlight distinct behaviors of three Transformer models.

Copied to clipboard

Jiasheng Zhang, Xikai Liu, Xinyi Lai, Yan Gao, Shusen Wang, Yao Hu, Yiqing Lin

Pre-Training to Learn in Context (2023.acl-long)

Challenge:	Named Entity Recognition (NER) tasks are a fundamental task of natural language processing (NLP).
Approach:	They propose a text-to-text framework for Few-Shot Named Entity Recognition (NER) that employs instruction finetuning and auxiliary tasks to enhance the model's understanding of entity types in the overall semantic context of a sentence.
Outcome:	The proposed framework outperforms existing Few-Shot NER methods and remains competitive with state-of-the-art NER algorithms.

Copied to clipboard

Yuxian Gu, Li Dong, Furu Wei, Minlie Huang

POE: Process of Elimination for Multiple Choice Reasoning (2023.emnlp-main)

Challenge:	Pre-trained language models are not explicitly trained to learn in context.
Approach:	They propose a framework to enhance in-context learning by pre-training language models on a large collection of "intrinsic tasks" they evaluate the in-constitution learning performance of the model trained with PICL on seven widely-used text classification datasets and the Super-NaturalInstrctions benchmark .
Outcome:	The proposed framework outperforms larger language models with nearly 4x parameters on seven widely-used datasets and the Super-NaturalInstrctions benchmark.

Copied to clipboard

Chenkai Ma, Xinya Du

HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Current language models perform well on multiple choice reasoning tasks, but the options are not treated equally.
Approach:	They propose a two-step scoring method that scores options and masks them to make the final prediction from the remaining options.
Outcome:	The proposed method is especially performant on logical reasoning tasks.

Copied to clipboard

Yihan Jiao, Zhehao Tan, Dan Yang, Duolin Sun, Jie Feng, Yue Shen, Jian Wang, Peng Wei

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021.emnlp-main)

Challenge:	In-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to inconsistent document quality and retrieval system imperfections.
Approach:	They propose that RAG models should possess three progressively hierarchical abilities: (1) Filtering: the ability to select relevant information; (2) Combination: the capability to combine semantic information across paragraphs; (3) RAG-specific reasoning: the capacity to further process external knowledge using internal knowledge.
Outcome:	Experiments show that the proposed method significantly improves the model’s open-book examination capability on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA.

Copied to clipboard

Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, Suk Hyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Donghoon Ham, Dongju Park, Min Young Lee, Jaewook Kang, Inho Kang, Jung-Woo Ha, Woomyoung Park, Nako Sung

Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models (2023.findings-acl)

Challenge:	GPT-3 has been used to train large-scale language models on hundreds of billion scale data.
Approach:	They propose a Korean variant of GPT-3 that uses Korean tokens to train in-context models.
Outcome:	The proposed method shows state-of-the-art zero-shot and few-shot learning on downstream tasks in Korean.

Copied to clipboard

Fengzhu Zeng, Wei Gao

Are Emergent Abilities in Large Language Models just In-Context Learning? (2024.acl-long)

Challenge:	Existing methods for few-shot and zero-shot fact verification require a large set of training data.
Approach:	They propose a method to prompt pre-trained language models to be consistent to improve the factuality assessment capability of PLMs.
Outcome:	The proposed method outperforms state-of-the-art few-shot fact verification models with a small number of unlabeled instances on zero-shot verification.

Copied to clipboard

Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, Iryna Gurevych

Narrative Style and the Spread of Health Misinformation on Twitter (2023.findings-emnlp)

Challenge:	Large language models have been claimed to acquire certain capabilities without having been specifically trained on them.
Approach:	They propose a theory that explains emergent abilities by taking into account their potential confounding factors and rigorously substantiate this theory through over 1000 experiments.
Outcome:	The proposed theory proves that emergent abilities are not truly emergental, but result from a combination of in-context learning, model memory, and linguistic knowledge.

Copied to clipboard

Achyutarama Ganti, Eslam Ali Hassan Hussein, Steven Wilson, Zexin Ma, Xinyan Zhao

Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning (2025.findings-naacl)

Challenge:	Using a narrative style is an effective way to communicate health information on and off social media platforms.
Approach:	They annotate health misinformation tweets and classify them into narrative and non-narrative . they then use supervised fine-tuning and in-context learning to detect narratives .
Outcome:	The proposed model analyzes health misinformation tweets and finds that narrative use is linked to increased tweet engagement and can lead to increased misinformation use.

Copied to clipboard

Joy Crosbie, Ekaterina Shutova

Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning (2023.findings-acl)

Challenge:	Large language models have shown a remarkable ability to learn and perform complex tasks through in-context learning (ICL).
Approach:	They analyse two state-of-the-art models, Llama-3-8B and InternLM2-20B on abstract pattern recognition and NLP tasks.
Outcome:	The proposed model can perform up to 32% better than previous models on abstract pattern recognition and NLP tasks.

Copied to clipboard

Ruixiang Tang, Dehan Kong, Longtao Huang, Hui Xue

Zero-shot Generative Linguistic Steganography (2024.naacl-long)

Challenge:	Large language models (LLMs) have shown great potential for in-context learning, but their robustness and performance on downstream tasks remains limited.
Approach:	They propose to examine the reliance of LLMs on shortcuts or spurious correlations within prompts for downstream tasks and find larger models are more likely to utilize shortcuts in prompts during inference.
Outcome:	The proposed model is “lazy learner” and more likely to use shortcuts in prompts during inference.

Copied to clipboard

Ke Lin, Yiyang Luo, Zijian Zhang, Luo Ping

Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning (2025.naacl-long)

Challenge:	Generative linguistic steganography attempts to hide secret messages into covertext . previous studies focused on the statistical differences between the covertext and stegotext - however, ill-formed stegotas can readily be identified by humans .
Approach:	They propose a zero-shot approach based on in-context learning for linguistic steganography to achieve better perceptual and statistical imperceptibility.
Outcome:	The proposed method produces 1.926 more innocent and intelligible stegotext than any other method.

Copied to clipboard

Rujing Yao, Yang Wu, Chenghao Wang, Jingwei Xiong, Fang Wang, Xiaozhong Liu

Can We Edit Factual Knowledge by In-Context Learning? (2023.emnlp-main)

Challenge:	Existing approaches to large language models focus on semantic similarity, neglecting the intricate logical structures and reasoning essential for addressing complex legal issues.
Approach:	They propose a Logical-Semantic Integration Model (LSIM) that bridges semantic and logical coherence and a supervised framework that integrates semantic features with in-context learning.
Outcome:	The proposed framework significantly improves accuracy and reliability on a real-world legal QA dataset.

Copied to clipboard

Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang

How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning? (2024.emnlp-main)

Challenge:	In-context knowledge editing (IKE) is a new paradigm for NLP research that can be applied to large language models with tens or hundreds of parameters.
Approach:	They propose to use in-context knowledge editing (IKE) without gradient updating to edit factual knowledge without a gradient update.
Outcome:	The proposed method achieves a competitive success rate compared to gradient-based methods on GPT-J but with fewer side effects.

Copied to clipboard

Yang Luo, Zangwei Zheng, Zirui Zhu, Yang You

In-Context Learning May Not Elicit Trustworthy Reasoning: A-Not-B Errors in Pretrained Language Models (2024.findings-emnlp)

Challenge:	MLLMs have significant capabilities for multimodal in-context learning, but their effectiveness hinges on the appropriate selection of in-constext examples.
Approach:	They propose a supervised MLLM prompt retriever that leverages a trained retriever based on MLML's confidence to select examples, which enhances multimodal in-context learning efficiency.
Outcome:	The proposed method is validated through extensive testing across three different tasks and demonstrates its effectiveness.

Copied to clipboard

Pengrui Han, Peiyang Song, Haofei Yu, Jiaxuan You

Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning (2025.findings-acl)

Challenge:	Recent advances in artificial intelligence have led to the creation of highly capable large language models (LLMs) that can perform tasks in a human-like manner, but lack infant-level cognitive abilities in certain areas.
Approach:	They designed a text-based multi-choice QA scenario similar to the A-Not-B error to test their inhibitory control abilities.
Outcome:	The proposed model shows that state-of-the-art LLMs perform well with in-context learning but make errors and show a drop of as many as 83.3% in reasoning tasks when the context changes trivially.

Copied to clipboard

Yilei Tu, Andrew Xue, Freda Shi

Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning (2025.findings-naacl)

Challenge:	In-context learning (ICL) is a widely adopted technique for learning large language models . however, there is little systematic understanding of when and why it works well .
Approach:	They analyze multilingual in-context learning using demonstrations in HRLs to enhance cross-lingual transfer.
Outcome:	The proposed method outperforms English-only models on high-resource languages . the study shows that the presence of irrelevant non-English sentences in the prompt yields measurable gains .

Copied to clipboard

Hyundong Justin Cho, Karishma Sharma, Nicolaas Paul Jedema, Leonardo F. R. Ribeiro, Jonathan May, Alessandro Moschitti

Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again (2022.findings-emnlp)

Challenge:	Language models are biased towards generic outputs as they are trained to align to an aggregate preference to be generally useful.
Approach:	They propose a tuning-free method that personalizes language models for text generation tasks with fewer than 10 examples per user.
Outcome:	The proposed method achieves favorable win rates on pairwise comparisons with the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles.

Copied to clipboard

Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun, Yu Su

Active Learning Principles for In-Context Learning with Large Language Models (2023.findings-emnlp)

Challenge:	Large pre-trained language models (PLMs) such as GPT-3 have shown strong in-context learning capabilities, which are appealing for domains such as biomedicine that feature high and diverse demands of language technologies but also high data annotation costs.
Approach:	They propose to compare the few-shot performance of GPT-3 in-context learning with fine-tuning smaller (i.e., BERT-sized) PLMs on two representative biomedical information extraction tasks: named entity recognition and relation extraction.
Outcome:	The proposed model underperforms on two representative biomedical information extraction tasks.

Copied to clipboard

Katerina Margatina, Timo Schick, Nikolaos Aletras, Jane Dwivedi-Yu

Can we teach language models to gloss endangered languages? (2024.findings-emnlp)

Challenge:	In-context learning has significantly enhanced predictive performance in few-shot learning settings.
Approach:	They propose to use pool-based Active Learning to identify the most informative demonstrations for few-shot learning over a single iteration to identify best demonstrations.
Outcome:	The proposed model outperforms all other methods, including random sampling, in the analysis of 24 classification and multi-choice tasks.

Copied to clipboard

Michael Ginn, Mans Hulden, Alexis Palmer

Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations (2024.acl-long)

Challenge:	Prior research has explored statistical and neural methods for automatically producing IGT.
Approach:	They propose to use in-context learning to generate interlinear glossed text . they propose to employ supervised learning to select examples to provide in-text .
Outcome:	The proposed methods beat standard transformer baselines, despite requiring no training at all.

Copied to clipboard

Yisong Miao, Hongfu Liu, Wenqiang Lei, Nancy Chen, Min-Yen Kan

Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning (2025.naacl-long)

Challenge:	Discursive Socratic Questioning (DISQ) assesses a model's understanding of discourse relations by requiring systematic accuracy over multiple questions.
Approach:	They propose a method that evaluates faithfulness of understanding discourse based on question answering.
Outcome:	The proposed method evaluates the faithfulness of understanding discourse based on question answering.

Copied to clipboard

Zixiao Zhu, Zijian Feng, Hanzhang Zhou, Junlang Qian, Kezhi Mao

Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (2023.findings-acl)

Challenge:	Effective organization of in-context learning (ICL) demonstrations is key to improving the quality of large language models (LLMs).
Approach:	They propose a logit separability-based method that integrates multiple class-related words into each sample-label pair to improve LLM understanding.
Outcome:	The proposed method improves ICL performance by providing clearer instructions and richer label information.

Copied to clipboard

Brendan King, Jeffrey Flanigan

EvoPrompt: Evolving Prompts for Enhanced Zero-Shot Named Entity Recognition with Large Language Models (2025.coling-main)

Challenge:	Recent work has demonstrated that in-context learning for dialogue state tracking outperforms training methods in the few-shot setting.
Approach:	They propose a method for in-context learning for dialogue state tracking that takes into account probabilities of competing surface forms and produces a more accurate dialogue state prediction.
Outcome:	The proposed method outperforms trained methods in the few-shot setting and requires little data and zero parameter updates.

Copied to clipboard

Zeliang Tong, Zhuojun Ding, Wei Wei

Learning Task Representations from In-Context Learning (2025.findings-acl)

Challenge:	Named Entity Recognition (NER) is a low-resource task that requires supervised learning, but practical scenarios lack annotated data.
Approach:	They propose an Evolving Prompts framework that guides the model to better address these issues through continuous prompt refinement.
Outcome:	The proposed framework shows consistent performance improvements on four benchmarks.

Copied to clipboard

Baturay Saglam, Xinyang Hu, Zhuoran Yang, Dionysis Kalogerias, Amin Karbasi

ARISE: Iterative Rule Induction and Synthetic Data Generation for Text Classification (2025.findings-naacl)

Challenge:	Existing methods for generalizing tasks to modalities beyond text fail to generalize effectively to linguistic tasks.
Approach:	They propose a method for encoding task information in ICL prompts as a function of attention heads within the transformer architecture.
Outcome:	The proposed method extracts task-specific information from in-context demonstrations and excels in both text and regression tasks.

Copied to clipboard

Yaswanth M, Vaibhav Singh, Ayush Maheshwari, Amrith Krishna, Ganesh Ramakrishnan

Cross-Lingual Learning vs. Low-Resource Fine-Tuning: A Case Study with Fact-Checking in Turkish (2024.lrec-main)

Challenge:	Existing frameworks for large language models (LLMs) generate high-quality synthetic data that can be used to supplement training data or surpass crowd-sourced annotations.
Approach:	They propose a framework that iteratively induces rules and generates synthetic data for text classification.
Outcome:	The proposed framework outperforms existing models on in-context learning and fine-tuning settings by using augmented data.

Copied to clipboard

Recep Firat Cekinel, Çağrı Çöltekin, Pinar Karagoz

CrossTune: Black-Box Few-Shot Classification with Label Enhancement (2024.lrec-main)

Challenge:	Currently, most of the research on misinformation is focused on the English language . however, there is a scarcity of datasets for other languages, including Turkish .
Approach:	They propose a dataset that spans multiple domains and incorporates evidence from three Turkish fact-checking organizations.
Outcome:	The proposed dataset has the potential to advance research in the Turkish language.

Copied to clipboard

Danqing Luo, Chen Zhang, Yan Zhang, Haizhou Li

Enough Coin Flips Can Make LLMs Act Bayesian (2025.acl-long)

Challenge:	Training large-scale language models requires substantial computation resources . current research focuses on adapting black-box models to downstream tasks using prompt optimization .
Approach:	They propose a label-enhanced cross-attention network called CrossTune to improve the generalization of the model.
Outcome:	The proposed approach outperforms the state-of-the-art black-box tuning method by 5.7% on average.

Copied to clipboard

Ritwik Gupta, Rodolfo Corona, Jiaxin Ge, Eric Wang, Dan Klein, Trevor Darrell, David M. Chan

The Unreasonable Effectiveness of Easy Training Data for Hard Tasks (2024.acl-long)

Challenge:	Large language models exhibit the ability to generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning.
Approach:	They investigate whether large language models use in-context learning to generalize given few-shot examples in their input prompt.
Outcome:	The proposed model can generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning.

Copied to clipboard

Peter Hase, Mohit Bansal, Peter Clark, Sarah Wiegreffe

Speech-based Slot Filling using Large Language Models (2024.findings-acl)

Challenge:	Existing pretrained language models perform well on hard data, but hard data is noisier and costlier to collect.
Approach:	They propose to use in-context learning, linear classifier heads, and QLoRA to show that pretrained language models generalize relatively well from easy to hard data.
Outcome:	The proposed model generalizes well from easy to hard data even better than oracle models finetuned on hard data.

Copied to clipboard

Guangzhi Sun, Shutong Feng, Dongcheng Jiang, Chao Zhang, Milica Gasic, Phil Woodland

On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model (2022.naacl-main)

Challenge:	Recent advances in large language models (LLMs) have shown an unprecedented ability across various language tasks.
Approach:	They propose to use prompts and LoRA fine-tuning to improve slot filling robustness . they propose a linearised knowledge injection scheme to integrate dynamic external knowledge into LLMs.
Outcome:	The proposed model improves slot filling with noisy ASR transcriptions with 6.7% and 17.6% absolute SLU-F1 improvements compared to a fully fine-tuned Flan-T5-XL model.

Copied to clipboard

Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-Woo Ha, Nako Sung

Fact-Checking Complex Claims with Program-Guided Reasoning (2023.acl-long)

Challenge:	Recent studies on large-scale in-context language models have reported successful in-const zero- and few-shot learning ability.
Approach:	They investigate the effects of the pretraining corpus on in-context learning in a Korean-centric model.
Outcome:	The study shows that pretraining corpus size does not determine in-context learning ability . the findings suggest that in-constext learning is not always competitive .

Copied to clipboard

Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, Preslav Nakov

Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-Making (2025.emnlp-main)

Challenge:	Fact-checking real-world claims often requires collecting multiple pieces of evidence and complex multi-step reasoning.
Approach:	They propose a novel fact-checking model that decomposes complex claims into simpler sub-tasks that can be solved using a shared library of specialized functions.
Outcome:	The proposed model outperforms seven baselines on two fact-checking datasets and has explicit output programs that benefit human debugging.

Copied to clipboard

Yuanjun Feng, Vivek Choudhary, Yash Raj Shrestha

Challenge:	Large language models (LLMs) are increasingly used for social science simulations . however, most evaluations focus on task optimality rather than variability and adaptation characteristic of human decision-making.
Approach:	They propose a process-oriented evaluation framework with progressive interventions to evaluate two economics tasks using large language models.
Outcome:	The proposed evaluation framework targets two economic tasks with progressive interventions.

Revealing the Parallel Multilingual Learning within Large Language Models (2024.emnlp-main)

Copied to clipboard

Yongyu Mu, Peinan Feng, Zhiquan Cao, Yuzhang Wu, Bei Li, Chenglong Wang, Tong Xiao, Kai Song, Tongran Liu, Chunliang Zhang, JingBo Zhu

Contrastive Learning for Prompt-based Few-shot Language Learners (2022.naacl-main)

Challenge:	Large language models (LLMs) can handle multilingual and cross-lingual text within a single input; however, previous studies focusing on using English as the pivot language to enhance language understanding and reasoning focus on using multiple languages.
Approach:	They propose to use parallel multilingual input to enhance the model's comprehension of the input and to examine how multilingual processing affects prediction.
Outcome:	The proposed model can handle multilingual and cross-lingual text within a single input, but previous studies focused on using English as the pivot language to enhance language understanding and reasoning.

Copied to clipboard

Yiren Jian, Chongyang Gao, Soroush Vosoughi

Ada-Instruct: Adapting Instruction Generators for Complex Reasoning (2024.findings-emnlp)

Challenge:	a recent study has shown that GPT-3 fine-tuning models with limited examples is effective . a contrastive learning framework clusters inputs from the same class under different augmented “views” and repels those from different classes.
Approach:	They propose a supervised contrastive framework that clusters inputs from the same class under different augmented "views" they combine a contrastive loss with the standard masked language modeling loss in prompt-based few-shot learners .
Outcome:	The proposed framework improves on the state-of-the-art methods in a diverse set of 15 language tasks.

Copied to clipboard

Wanyun Cui, Qianle Wang

Finding Support Examples for In-Context Learning (2023.findings-emnlp)

Challenge:	Existing methods for generating instructions from a few initial samples with in-context learning are lacking in generating complex instructions of length 100.
Approach:	They propose an adaptive instruction generator developed through fine-tuning that generates long, intricate, and distributionally consistent instructions.
Outcome:	The proposed method generates long, intricate, and distributionally consistent instructions with ten examples.

Copied to clipboard

Xiaonan Li, Xipeng Qiu

Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models (2023.findings-emnlp)

Challenge:	In-context learning is a new learning paradigm where a language model observes a few examples and directly outputs the test input’s prediction.
Approach:	They propose a method to find “support examples” for in-context learning by filtering a training dataset and a progressive filtering process to filter out uninformative examples.
Outcome:	The proposed method outperforms baselines and shows that each component contributes critically to the improvements.

Copied to clipboard

Haoran Wang, Kai Shu

Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models (2025.naacl-long)

Challenge:	Existing claims verification models rely on annotated data, which is expensive to create at a large scale.
Approach:	They propose a model that can verify complex claims without annotated data . they leverage the in-context learning ability of Large Language Models to translate a claim into a First-Order-Logic clause .
Outcome:	The proposed model outperforms baseline models on three datasets . it performs well on the datasets, and the results are published online.

Copied to clipboard

Aliakbar Nafar, K. Brent Venable, Parisa Kordjamshidi

Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)

Challenge:	Existing studies on in-context learning mechanisms are not consistent . current research identifies two main approaches to explain the ICL mechanism .
Approach:	They propose a framework for evaluating in-context learning mechanisms by focusing on regression tasks.
Outcome:	The proposed framework can solve regression problems and then measure the extent to which the LLM retrieves its internal knowledge versus learning from in-context examples.

Copied to clipboard

Zhiyuan He, Huiqiang Jiang, Zilong Wang, Yuqing Yang, Luna Qiu, Lili Qiu

Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning (2024.emnlp-main)

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence.
Approach:	They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself.
Outcome:	The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios.

Copied to clipboard

Jingyu Hu, Weiru Liu, Mengnan Du

Analysing The Impact of Sequence Composition on Language Model Pre-Training (2024.acl-long)

Challenge:	Recent studies highlight the effectiveness of using in-context learning (ICL) to steer large language models in processing tabular data.
Approach:	They propose a method that uses clustering and evolutionary strategies to curate a representative sample set from training data.
Outcome:	The proposed method significantly improves fairness across various metrics, showing its efficacy in real-world scenarios.

Copied to clipboard

Yu Zhao, Yuanbin Qu, Konrad Staniszewski, Szymon Tworkowski, Wei Liu, Piotr Miłoś, Yuxiang Wu, Pasquale Minervini

Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (2024.findings-acl)

Challenge:	Existing studies show that pretraining sequence composition strategy can lead to distracting information from previous documents.
Approach:	They propose to use a sequence construction method to concatenate documents into fixed-length sequences to compute the likelihood of each token given its context.
Outcome:	The proposed method can improve in-context learning, knowledge memorisation and context utilisation without sacrificing efficiency.

Copied to clipboard

Lvxue Li, Jiaqi Chen, Xinyu Lu, Yaojie Lu, Hongyu Lin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun

Harnessing the Power of Large Language Models for Empathetic Response Generation: Empirical Investigations and Improvements (2023.findings-emnlp)

Challenge:	In-context learning (ICL) has gained considerable attention due to its data efficiency and task adaptability.
Approach:	They propose to de-biase demonstration bias in in-context learning by focusing on semantic ambiguity induced by demonstrations and reducing the semantic hazard.
Outcome:	The proposed methods significantly improve performance on six datasets.

Copied to clipboard

Yushan Qian, Weinan Zhang, Ting Liu

In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)

Challenge:	Empathetic dialogue is an essential part of building harmonious social relationships and contributes to the development of a helpful AI.
Approach:	They propose three methods to improve the performance of large language models (LLMs) they propose semantically similar in-context learning, two-stage interactive generation and combination with the knowledge base.
Outcome:	The proposed methods achieve state-of-the-art in automatic and human evaluations and the possibility of GPT-4 simulating human evaluators.

Copied to clipboard

Chengwei Qin, Aston Zhang, Chen Chen, Anirudh Dagar, Wenming Ye

The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis (2024.findings-acl)

Challenge:	Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific.
Approach:	They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations.
Outcome:	The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks.

Copied to clipboard

Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba Alabi, Xiaoyu Shen, Dietrich Klakow, Marius Mosbach

From Chaos to Clarity: Claim Normalization to Empower Fact-Checking (2023.findings-emnlp)

Challenge:	In-context learning is a popular inference strategy where large language models solve a task using only a few labeled demonstrations without updating the model parameters.
Approach:	They conduct multidimensional analysis of multilingual in-context learning using 5 models from different model families and 9 datasets covering classification and generation tasks.
Outcome:	The results show that demonstrations vary significantly across models, tasks, and languages.

Copied to clipboard

Megha Sundriyal, Tanmoy Chakraborty, Preslav Nakov

An Empirical Study of In-context Learning in LLMs for Machine Translation (2024.findings-acl)

Challenge:	Social media posts are noisy and pervasive, resulting in difficult to identify precise and prominent claims that require verification.
Approach:	They propose a task called Claim Normalization that decomposes complex and noisy social media posts into more straightforward and understandable forms, termed normalized claims.
Outcome:	The proposed model outperforms baselines across evaluation measures and errors.

Copied to clipboard

Pranjal Chitale, Jay Gala, Raj Dabre

Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training (2025.naacl-long)

Challenge:	Recent studies focus on optimizing translation quality, with limited attention to understanding specific aspects of ICL that influence the said quality.
Approach:	They conduct the first of its kind, exhaustive study of in-context learning for machine translation (MT) they establish that ICL is primarily example-driven and not instruction-driven .
Outcome:	The proposed model is based on examples and not instruction-driven learning.

Copied to clipboard

Deven Mahesh Mistry, Anooshka Bajaj, Yash Aggarwal, Sahaj Singh Maini, Zoran Tiganj

From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning (2023.emnlp-main)

Challenge:	Existing studies have shown that attention heads have a temporal induction property that allows them to learn and reproduce sequences of tokens.
Approach:	They analyze attention heads and transformer outputs to examine in-context temporal biases . they find that transformer output has a tendency toward in-constext serial recall .
Outcome:	The findings shed light on similarities and differences between LLMs and human memory and learning.

Copied to clipboard

Zheyuan Zhang, Shane Storks, Fengyuan Hu, Sungryull Sohn, Moontae Lee, Honglak Lee, Joyce Chai

Universal Self-Adaptive Prompting (2023.emnlp-main)

Challenge:	Pre-trained language models have shown impressive performance in various language tasks, but are prone to spurious correlations and illusory information.
Approach:	They propose to use pre-trained language models to justify decisions with formalized, coherent reasoning chains.
Outcome:	The proposed strategies improve coherence of rationalizations yielding state-of-the-art results on Tiered Reasoning for Intuitive Physics (TRIP).

Copied to clipboard

Xingchen Wan, Ruoxi Sun, Hootan Nakhost, Hanjun Dai, Julian Eisenschlos, Sercan Arik, Tomas Pfister

EduMARS: Can Vision-Language Models Grade Like Teachers? Benchmarking Multimodal, Rubric-Based Assessment on Chinese K-12 Answers (2026.findings-acl)

Challenge:	a hallmark of modern large language models is their impressive general zero-shot and few-shot abilities . however, zero- shot performances are weaker due to the lack of guidance and the difficulty of applying existing automatic prompt design methods in general tasks.
Approach:	They propose an automatic prompt design approach specifically tailored for zero-shot learning that categorizes a possible NLP task into one of three possible task types and then uses a selector to select the most suitable queries and zero- shot model-generated responses as pseudo-demonstrations.
Outcome:	The proposed approach is able to generalize ICL to zero-shot learning tasks while also allowing for a more efficient and efficient prompt design.

Copied to clipboard

Xuan Zhao, Jiashun Chen, Wanting xu, Huiyuan Yan, Chaowei Fang, Xing Wei

PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization (2025.acl-long)

Challenge:	Existing benchmarks for automated grading of student work fail to evaluate real student responses . existing models fail to assess real student work, especially on cognitively demanding tasks .
Approach:	They propose a multimodal benchmark for rubric-aligned evaluation of real Chinese K-12 student answers.
Outcome:	The proposed model improves performance and interpretability of existing models on EduMARS . existing models fail to perform on real-world, cognitively demanding tasks, authors say .

Copied to clipboard

Yidan Wang, Yanan Cao, Yubing Ren, Fang Fang, Zheng Lin, Binxing Fang

Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing (2023.findings-emnlp)

Challenge:	Existing methods to evaluate privacy leakage in LLMs use memorized prefixes or simple instructions to extract data, which well-aligned models can easily block.
Approach:	They propose a framework targeting Personally Identifiable Information (PII) that uses in-context learning to build a privacy context and iteratively updates it with three gradient-based strategies to elicit target PII.
Outcome:	The proposed framework outperforms baseline methods and achieves state-of-the-art (SoTA) results on four white-box and two black-box LLMs.

Copied to clipboard

Shufan Wang, Sébastien Jean, Sailik Sengupta, James Gung, Nikolaos Pappas, Yi Zhang

Revisiting Demonstration Selection Strategies in In-Context Learning (2024.acl-long)

Challenge:	In task-oriented semantic parsing, the system aims to translate users’ utterances in natural language to machine-interpretable programs (API calls) However, Large Language Models (LLMs) are known to hallucinate and therefore pose a formidable challenge in constraining generated content.
Approach:	They propose to use large language models to translate user's utterances to machine-interpretable programs (API calls) they identify constraints violations in task-oriented utterrances and define fine-grained metrics that complement traditional ones.
Outcome:	The proposed methods reduce constraints violations and improve quality of the generated API calls, but require careful consideration given their implementation complexity and latency.

Copied to clipboard

Keqin Peng, Liang Ding, Yancheng Yuan, Xuebo Liu, Min Zhang, Yuanxin Ouyang, Dacheng Tao

The Mystery of Compositional Generalization in Graph-based Generative Commonsense Reasoning (2024.findings-emnlp)

Challenge:	Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL).
Approach:	They propose a data- and model-dependent method to select models using in-context learning, TopK + ConE, and propose unified explanations for the effectiveness of previous methods.
Outcome:	The proposed method improves language understanding and generation tasks with different model scales.

Copied to clipboard

Xiyan Fu, Anette Frank

Cognate Detection for Historical Language Reconstruction of Proto-Sabean Languages: the Case of Ge’ez, Tigrinya, and Amharic (2025.coling-main)

Challenge:	Existing studies have found that LLMs are limited in scenarios that require generalization abilities, such as out-of-domain tasks.
Approach:	They propose a Compositional Generalization Challenge for Graph-based Commonsense Reasoning that requires models to generate a natural sentence based on given concepts and a reasoning graph.
Outcome:	The proposed framework is based on seven well-known LLMs and shows that they struggle in compositional generalization.

Copied to clipboard

Elleni Sisay Temesgen, Hellina Hailu Nigatu, Fitsum Assamnew Andargie

Challenge:	As languages evolve, we risk losing ancestral languages.
Approach:	They propose to use cognates to reconstruct proto-languages from cognates in child languages that have likely evolved from the same word in the proto-linguistics.
Outcome:	The proposed method is based on automatic cognate detection and in-context learning with GPT-4o to generate the proto-language from the cognates and use Sequence-to-Sequence models.

Perspective Transition of Large Language Models for Solving Subjective Tasks (2025.findings-acl)

Copied to clipboard

Xiaolong Wang, Yuanchi Zhang, Ziyue Wang, Yuzhuang Xu, Fuwen Luo, Yile Wang, Peng Li, Yang Liu

Leveraging the Cross-Domain & Cross-Linguistic Corpus for Low Resource NMT: A Case Study On Bhili-Hindi-English Parallel Corpus (2025.findings-emnlp)

Challenge:	Large language models (LLMs) have revolutionized the field of natural language processing . performance of LLMs on subjective tasks is limited, authors say .
Approach:	They propose a method that allows LLMs to select between direct, role, and third-person perspectives for best way to solve corresponding subjective problem.
Outcome:	The proposed method outperforms widely used single fixed perspective based methods on 12 subjective tasks.

Copied to clipboard

Pooja Singh, Shashwat Bhardwaj, Vaibhav Sharma, Sandeep Kumar

EmpCRL: Controllable Empathetic Response Generation via In-Context Commonsense Reasoning and Reinforcement Learning (2024.lrec-main)

Challenge:	linguistic diversity of India poses significant machine translation challenges, authors say . underrepresented tribal languages like Bhili lack high-quality linguistic resources .
Approach:	They introduce a Bhili-Hindi-English Parallel Corpus, the first and largest parallel corpus worldwide . they evaluated a wide range of proprietary and open-source MLLMs on bidirectional translation tasks .
Outcome:	The proposed corpus spans critical domains such as education, administration, and news.

Copied to clipboard

Mingxiu Cai, Daling Wang, Shi Feng, Yifei Zhang

PICLe: Pseudo-annotations for In-Context Learning in Low-Resource Named Entity Detection (2025.naacl-long)

Challenge:	Existing studies lack the perception of fine-grained dialogue emotion propagation, and have limitations in reasoning about the intentions of users on cognition, which affect the quality of empathetic response.
Approach:	They propose to use commonsense reasoning and reinforcement learning to generate empathetic response based on in-context commonsensing and contextual reasoning to broaden cognitive boundaries.
Outcome:	The proposed model outperforms state-of-the-art models in automatic and human evaluation.

Copied to clipboard

Sepideh Mamooler, Syrielle Montariol, Alexander Mathis, Antoine Bosselut

Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy (2023.emnlp-main)

Challenge:	In-context learning is sensitive to the choice of demonstrations and can be used for tasks with few examples.
Approach:	They propose a framework for in-context learning with noisy, pseudo-annotated demonstrations . they annotate large quantities of demonstrations in a zero-shot first pass .
Outcome:	The proposed framework outperforms ICL on biomedical NED datasets with zero human-annotation.

Copied to clipboard

Sarah Wiegreffe, Matthew Finlayson, Oyvind Tafjord, Peter Clark, Ashish Sabharwal

What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023.findings-acl)

Challenge:	Pretrained language models (LMs) are used to discriminate on multiple-choice tasks that place probability mass on vocabulary tokens that aren’t among the given answer choices.
Approach:	They propose a mathematical formalism for SFC which allows us to quantify and bound its impact for the first time.
Outcome:	The proposed method eliminates the impact of SFC in the majority of instances.

Copied to clipboard

Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen

Catch Me If You Can? Not Yet: LLMs Still Struggle to Imitate the Implicit Writing Styles of Everyday Authors (2025.findings-emnlp)

Challenge:	Large language models (LLMs) can perform in-context learning (ICL) with only a few demonstrations, but its mechanisms are not well-understood.
Approach:	They characterize two ways in which LLMs leverage demonstrations to solve tasks with a few demonstrations.
Outcome:	The proposed model achieves non-trivial performance with only TR, and TR does not improve with larger models or more demonstrations.

Copied to clipboard

Zhengxiang Wang, Nafis Irtiza Tripto, Solha Park, Zhenzhen Li, Jiawei Zhou

Semantic Captioning: Benchmark Dataset and Graph-Aware Few-Shot In-Context Learning for SQL2Text (2025.coling-main)

Challenge:	Personal style is often subtle and implicit, making it difficult to specify through prompts yet essential for user-aligned generation.
Approach:	They evaluate LLMs' ability to imitate personal writing styles via in-context learning from user-authored samples.
Outcome:	The proposed model can imitate personal writing styles from a small number of user-authored samples.

Copied to clipboard

Ali Al Lawati, Jason Lucas, Prasenjit Mitra

Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)

Challenge:	Large Language Models (LLMs) have shown remarkable performance in various NLP tasks, including semantic parsing, which translates natural language into formal code representations.
Approach:	They propose a semantic captioning task to repurpose semantic parsing datasets for semantic captions.
Outcome:	The proposed model outperforms random selection and other methods by 39% on BLEU score.

Copied to clipboard

Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou

Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning (2024.findings-emnlp)

Challenge:	In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks.
Approach:	They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks.
Outcome:	The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show .

Copied to clipboard

Tianhui Zhang, Bei Peng, Danushka Bollegala

Few-Shot Natural Language to First-Order Logic Translation via Code Generation (2025.naacl-long)

Challenge:	Large Language Models (LLMs) have shown proficiency in enhancing the generation quality across various tasks without the need for any fine-tuning.
Approach:	They propose a method that diversifies the LLM generations while preserving their quality.
Outcome:	The proposed method can be used as training data to improve diversity in existing commonsense generators.

Copied to clipboard

Junnan Liu

Tutor-ICL: Guiding Large Language Models for Improved In-Context Learning Performance (2024.findings-emnlp)

Challenge:	Recent studies have focused on translation of natural language to first-order logical formula (NL-FOL) but these methods face challenges such as inconsistency between training and inference phases and data-intensive finetuning process.
Approach:	They propose a method for translating natural language into first-order logical formulas using code snippets.
Outcome:	The proposed method surpasses training-free baselines and is comparable to supervised models trained on the full training data.

Copied to clipboard

Ikhyun Cho, Gaeul Kwon, Julia Hockenmaier

LLM-Supported Natural Language to Bash Translation (2025.naacl-long)

Challenge:	In-context learning (ICL) is a dominant paradigm in natural language processing.
Approach:	They propose a prompting method for classification tasks using exemplar answers in a *comparative format' they also propose introducing a test instance before the exemplars to improve performance .
Outcome:	The proposed method achieves up to 13.76% increase in accuracy on classification tasks across decoder-only and encoder-decoder LLMs.

Copied to clipboard

Finnian Westenfelder, Erik Hemberg, Stephen Moskal, Una-May O’Reilly, Silviu Chiricescu

Enhancing Tool Retrieval with Iterative Feedback from Large Language Models (2024.findings-emnlp)

Challenge:	Using the natural language to Bash command (NL2SH) for command composition is difficult due to inaccurate test data and unreliable heuristics for determining the functional equivalence of Bash commands.
Approach:	They propose to use a heuristic to determine the functional equivalence of two Bash commands with 95% confidence, a 16% increase over previous heurs.
Outcome:	The proposed heuristic can determine the functional equivalence of two Bash commands with 95% confidence, a 16% increase over previous heurs.

Copied to clipboard

Qiancheng Xu, Yongqi Li, Heming Xia, Wenjie Li

CAPE: Context-Aware Personality Evaluation Framework for Large Language Models (2025.findings-emnlp)

Challenge:	Existing methods have shown that large language models can handle a certain amount of tools through in-context learning or fine-tuning.
Approach:	They propose to enhance tool retrieval with iterative feedback from the large language model by prompting the tool usage model to provide feedback for the tool retriever model in multi-round.
Outcome:	The proposed approach achieves advanced performance in both in-domain evaluation and out-of-domain assessment.

Copied to clipboard

Jivnesh Sandhan, Fei Cheng, Tushar Sandhan, Yugo Murawaki

In-context Examples Selection for Machine Translation (2023.findings-acl)

Challenge:	Existing studies use a context-free approach to assess humans . existing studies use the Disney World test, which ignores real-world applications .
Approach:	They propose a framework to assess personality traits in large language models . they use conversational history to quantify the consistency of LLM responses .
Outcome:	The proposed framework improves consistency of responses in large language models . it also shows that conversational history enhances consistency and personality shifts .

Copied to clipboard

Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, Marjan Ghazvininejad

Evaluating Code-Switching Translation with Large Language Models (2024.lrec-main)

Challenge:	Large-scale generative models can perform a wide range of NLP tasks using in-context learning.
Approach:	They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings.
Outcome:	The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets.

Copied to clipboard

Muhammad Huzaifah, Weihua Zheng, Nattapol Chanpaisit, Kui Wu

The Stochastic Parrot on LLM’s Shoulder: A Summative Assessment of Physical Concept Understanding (2025.naacl-long)

Challenge:	Recent advances in large language models (LLMs) have shown they can match or surpass finetuned models on many natural language processing tasks.
Approach:	They propose to use in-context learning and pivot translation to improve code-switching translation.
Outcome:	The proposed models show strong ability for cross-lingual understanding in a code-switching setting.

Copied to clipboard

Mo Yu, Lemao Liu, Junjie Wu, Tsz Ting Chung, Shunchi Zhang, Jiangnan Li, Dit-Yan Yeung, Jie Zhou

Challenge:	Recent years have witnessed remarkable advancements in large language models (LLMs) many researchers argue that LLMs may not * Equal contribution.
Approach:	They propose a task that summarises the memorization issue by using grid inputs that abstractly describe physical phenomena.
Outcome:	The proposed task alleviates the memorization issue by using grid-format inputs that abstractly describe physical phenomena.

Jailbreaking? One Step Is Enough! (2025.acl-long)

Copied to clipboard

Weixiong Zheng, Peijian Zeng, YiWei Li, Hongyan Wu, Nankai Lin, Junhao Chen, Aimin Yang, Yongmei Zhou

LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error (2024.acl-long)

Challenge:	Large language models (LLMs) excel in various tasks but remain vulnerable to jailbreak attacks, where adversaries manipulate prompts to generate harmful outputs.
Approach:	They propose a Reverse Embedded Defense Attack mechanism that disguises the attack intention as the "defense" intention against harmful content.
Outcome:	The proposed method outperforms existing methods on open-source and closed-source models and enables successful jailbreak in one iteration.

Copied to clipboard

Boshi Wang, Hao Fang, Jason Eisner, Benjamin Van Durme, Yu Su

Is In-Context Learning a Type of Error-Driven Learning? Evidence from the Inverse Frequency Effect in Structural Priming (2025.naacl-long)

Challenge:	Existing work on tool-augmented LLMs focuses on the broad coverage of tools and the flexibility of adding new tools.
Approach:	They propose a biologically inspired method for tool-augmented LLMs that orchestrates three key mechanisms for successful tool use behaviors in the biological system: trial and error, imagination, and memory.
Outcome:	The proposed method improves tool learning for LLMs under both in-context learning and fine-tuning settings, bringing a boost of 46.7% to Mistral-Instruct-7B and outperforms GPT-4.

Copied to clipboard

Zhenghao Zhou, Robert Frank, R. Thomas McCoy

Psycholinguistics

Challenge:	Recent pre-trained large language models have shown the capacity to perform in-context learning (ICL) this capability could provide a way to bridge the divide between language models and humans.
Approach:	They propose a new way of diagnosing whether ICL is error-driven learning . they simulated structural priming with ICL and found the effect was stronger .
Outcome:	The proposed method is based on the inverse frequency effect (IFE) phenomenon is similar to error-driven learning in large language models .

StraGo: Harnessing Strategic Guidance for Prompt Optimization (2024.findings-emnlp)

Copied to clipboard

Yurong Wu, Yan Gao, Bin Zhu, Zineng Zhou, Xiaodi Sun, Sheng Yang, Jian-Guang Lou, Zhiming Ding, Linjun Yang

Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners (2024.acl-long)

Challenge:	Existing methods for prompt optimization often lead to prompt drifting, wherein newly generated prompts canadversely impact previously successful cases while addressing failures.
Approach:	They propose a method to mitigate prompt drifting by integrating in-context learning to formulate specific, actionable strategies for prompt optimization.
Outcome:	The proposed approach mitigates prompt drifting by leveraging insights from both successful and failed cases to identify critical factors for achieving optimization objectives.

Copied to clipboard

Rongjie Huang, Chunlei Zhang, Yongqi Wang, Dongchao Yang, Jinchuan Tian, Zhenhui Ye, Luping Liu, Zehan Wang, Ziyue Jiang, Xuankai Chang, Jiatong Shi, Chao Weng, Zhou Zhao, Dong Yu

C-ICL: Contrastive In-context Learning for Information Extraction (2024.findings-emnlp)

Challenge:	Large language models (LLMs) have been used for general-purpose interfaces across multiple tasks and languages.
Approach:	They propose to use large language models as a general-purpose interface across multiple tasks and languages.
Outcome:	The proposed model performs better on 200K hours of 6-language data for voice generation applications.

Copied to clipboard

Ying Mo, Jiahao Liu, Jian Yang, Qifan Wang, Shun Zhang, Jingang Wang, Zhoujun Li

Just Adjust One Prompt: Enhancing In-Context Dialogue Scoring via Constructing the Optimal Subgraph of Demonstrations and Prompts (2023.emnlp-main)

Challenge:	Existing methods for in-context learning with large language models focus on using correct or negative examples, ignoring the potential value of incorrect or negative samples.
Approach:	They propose a few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations.
Outcome:	The proposed technique outperforms previous few-shot in-context learning methods on a broad spectrum of related tasks.

Copied to clipboard

Jiashu Pu, Ling Cheng, Lu Fan, Tangjie Lv, Rongsheng Zhang

HomeBench: Evaluating LLMs in Smart Homes with Valid and Invalid Instructions Across Single and Multiple Devices (2025.acl-long)

Challenge:	Using large language models as chatbots can cause hallucinations and lack of empathy, authors report . a dimension-agnostic scoring method is proposed to improve the performance of chatbot performance .
Approach:	They propose a dimension-agnostic scoring method that leverages in-context learning . they propose to automatically generate prompts and then request the LLM multiple times .
Outcome:	The proposed method outperforms baselines on five datasets.

Copied to clipboard

Silin Li, Yuhang Guo, Jiashu Yao, Zeming Liu, Haifeng Wang

Challenge:	Existing state-of-the-art LLMs cannot perform well in situations where instructions are invalid or multiple devices are involved.
Approach:	They propose to integrate large language models into smart home assistants by enhancing their ability to accurately understand user needs and respond appropriately.
Outcome:	The proposed dataset is the first with valid and invalid instructions across devices . it achieves only 0.0% success rate in the scenario of invalid multi-device instructions .

Automatic Mathematic In-Context Example Generation for LLM Using Multi-Modal Consistency (2025.coling-main)

Copied to clipboard

Jaeseong Lee, Wei Yang, Gopal Gupta, Shiyi Wei

Integrating Visual Modalities with Large Language Models for Mental Health Support (2025.coling-main)

Challenge:	Existing methods for in-context learning require annotated datasets, resulting in higher computational costs and lower quality examples.
Approach:	They propose a framework that automatically generates high-quality in-context examples to enhance LLMs’ mathematical reasoning.
Outcome:	Evaluated on four math problem datasets, the proposed framework outperforms baseline methods with LLM accuracy ranging from 87.0% to 99.3%.

Copied to clipboard

Zhouan Zhu, Shangfei Wang, Yuxin Wang, Jiaqiang Wu

Visual Data in NLP

Challenge:	Existing work of mental health support primarily utilizes unimodal textual data and fails to understand and respond to users’ emotional states comprehensively.
Approach:	They propose a framework that integrates multimodal inputs and counseling strategies to enhance the performance of Large Language Models (LLMs) This approach allows LLMs to generate more nuanced and supportive responses.
Outcome:	The proposed framework outperforms existing models and delivers more empathetic, coherent, and contextually relevant mental health support responses.

In-Context Learning with Long-Context Models: An In-Depth Exploration (2025.naacl-long)

Copied to clipboard

Amanda Bertsch, Maor Ivgi, Emily Xiao, Uri Alon, Jonathan Berant, Matthew R. Gormley, Graham Neubig

Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching (2025.findings-acl)

Challenge:	In-context learning is limited by context length, but it can be used for many tasks.
Approach:	They study the behavior of in-context learning at an extreme context length . example retrieval shows excellent performance at low context lengths but has diminished gains .
Outcome:	The proposed model can perform many tasks with reasonable accuracy when a few examples are provided in-context.

Copied to clipboard

Jianfei Zhang, Bei Li, Jun Bai, Rumei Li, Yanmeng Wang, Chenghua Lin, Wenge Rong

Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks (2024.acl-long)

Challenge:	In-Context Learning (ICL) empowers Large Language Models for rapid task adaptation without fine-tuning.
Approach:	They propose a method that aligns fine-tuning gradients between entire training set and selected examples to enable in-context learning and fine-uning.
Outcome:	The proposed method outperforms random selection on large LLMs from 4-shot to 128-shot scenarios across 9 datasets.

Copied to clipboard

Anwoy Chatterjee, Eshaan Tanwar, Subhabrata Dutta, Tanmoy Chakraborty

ParaICL: Towards Parallel In-Context Learning (2025.naacl-long)

Challenge:	Large Language Models (LLMs) have transformed NLP with their remarkable In-context Learning capabilities.
Approach:	They propose to use large language models to generalize from labeled examples of predefined tasks to novel tasks . they use biological neurons and the Transformer architecture to study the potential for information sharing across tasks.
Outcome:	The proposed model can generalize from labeled examples of predefined tasks to novel tasks despite no examples from the target task in the context.

Copied to clipboard

Xingxuan Li, Xuan-Phi Nguyen, Shafiq Joty, Lidong Bing

FaiMA: Feature-aware In-context Learning for Multi-domain Aspect-based Sentiment Analysis (2024.lrec-main)

Challenge:	Existing methods to improve ICL performance are limited by the length of the input context.
Approach:	They propose a method that utilizes all demonstration examples without exceeding the manageable context length.
Outcome:	The proposed method can be scaled up to integrate with existing methods.

Copied to clipboard

Songhua Yang, Xinke Jiang, Hanjie Zhao, Wenxuan Zeng, Hongde Liu, Yuxiang Jia

Sentiment Analysis

Challenge:	Existing methods for aspect-based sentiment analysis are limited and integrating with existing techniques is difficult.
Approach:	They propose a framework that utilizes in-context learning as a feature-aware mechanism that facilitates adaptive learning in multi-domain ABSA tasks.
Outcome:	The proposed framework achieves significant performance improvements in multiple domains compared to baselines, increasing F1 by 2.07% on average.

Active Example Selection for In-Context Learning (2022.emnlp-main)

Copied to clipboard

Yiming Zhang, Shi Feng, Chenhao Tan

FrameEOL: Semantic Frame Induction using Causal Language Models (2025.findings-emnlp)

Challenge:	In-context learning performance is unstable across samples of examples, suggesting the idiosyncrasies of how language models acquire information.
Approach:	They propose a reinforcement learning algorithm for identifying generalizable policies to select demonstration examples and propose 'in-context learning' performance can be highly unstable across samples of examples, suggesting the idiosyncrasies of how language models acquire information.
Outcome:	The proposed model can perform tasks with examples with a 5.8% improvement on GPT-2 and GPT-3, but the improvement diminishes on larger models, suggesting emerging capabilities of large language models.

Copied to clipboard

Chihiro Yano, Kosuke Yamada, Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda

Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing (2022.emnlp-main)

Challenge:	Semantic frame induction is the task of clustering frame-evoking words according to the semantic frames they evoke.
Approach:	They propose a prompt-based method for obtaining Frame Embeddings that outputs One frame-name as a Label .
Outcome:	The proposed method outperforms existing methods on English and Japanese datasets.

Copied to clipboard

Linlu Qiu, Peter Shaw, Panupong Pasupat, Tianze Shi, Jonathan Herzig, Emily Pitler, Fei Sha, Kristina Toutanova

Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs (2024.emnlp-main)

Challenge:	Pre-trained language models struggle on out-of-distribution compositional generalization . recent work shows considerable improvements on many NLP tasks from model scaling .
Approach:	They evaluate encoder-decoder models up to 11B parameters and decoder-only models up 540B parameters . they compare scaling curves for fine-tuning, prompt tuning, and in-context learning methods .
Outcome:	The proposed scaling methods improve compositional generalization on many tasks . fine-tuning generally has flat or negative scaling curves on out-of-distribution compositional . larger models are better at modeling the syntax of the output space, the study finds .

Copied to clipboard

Haritz Puerto, Martin Tutek, Somak Aditya, Xiaodan Zhu, Iryna Gurevych

HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation (2023.acl-long)

Challenge:	Recent prompting techniques have improved LLMs’ performance on various reasoning tasks, but there is little understanding of what triggers reasoning abilities in LLM in the inference stage.
Approach:	They propose a method that transforms a natural language problem into code and directly prompts the LLM using the generated code without resorting to external code execution.
Outcome:	The proposed method boosts multiple LLMs by 22.52 percentage points on GPT 3.5, 7.75 on Mixtral, and 16.78 on Mistral.

Copied to clipboard

Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew Peters

Estimating Large Language Model Capabilities without Labeled Test Data (2023.findings-emnlp)

Challenge:	Recent NLP models have shown the remarkable ability to generalise ‘zero-shot’ to new tasks using only natural language instructions as guidance.
Approach:	They introduce Hypernetworks for INstruction Tuning (HINT) which converts task instructions and examples into parameter-efficient modules inserted into an underlying model using a pretrained text encoder.
Outcome:	The proposed models outperform strong state-of-the-art models by over 10% when controlling for compute.

Copied to clipboard

Harvey Fu, Qinyuan Ye, Albert Xu, Xiang Ren, Robin Jia

Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning (2024.emnlp-main)

Challenge:	Large Language Models have shown impressive ability to perform in-context learning from only a few examples, but their accuracy varies widely from task to task.
Approach:	They propose a method that trains a meta-model using LLM confidence scores as features to perform ICL accuracy estimation.
Outcome:	The proposed method improves over baselines across 7 out of 12 settings and achieves the same accuracy as evaluating on 40 sampled examples per task.

Copied to clipboard

Shuai Zhao, Meihuizi Jia, Anh Tuan Luu, Fengjun Pan, Jinming Wen

LLMs Learn Task Heuristics from Demonstrations: A Heuristic-Driven Prompting Strategy for Document-Level Event Argument Extraction (2024.acl-long)

Challenge:	In-context learning has shown high efficacy in several NLP tasks, especially in few-shot settings.
Approach:	They propose a backdoor attack method that poisons demonstration examples and poisons the demonstration context, preserving the model's generality.
Outcome:	The proposed method can make models behave in alignment with predefined intentions without fine-tuning the model.

Copied to clipboard

Hanzhang Zhou, Junlang Qian, Zijian Feng, Lu Hui, Zixiao Zhu, Kezhi Mao

Reasoning Graph Enhanced Exemplars Retrieval for In-Context Learning (2025.coling-main)

Challenge:	In-context learning (ICL) is an emerging ability of large-scale labeled data for document-level event argument extraction (EAE).
Approach:	They propose an explicit heuristic-driven demonstration construction approach that emphasizes task heurs in document-level event argument extraction tasks.
Outcome:	The proposed method outperforms existing prompting methods and few-shot supervised learning methods on document-level EAE datasets.

Copied to clipboard

Yukang Lin, Bingchen Zhong, Shuoran Jiang, Joanna Siebert, Qingcai Chen

Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL (2025.coling-main)

Challenge:	Existing methods focus on semantic similarity between queries and candidate exemplars, while logical connections between reasoning steps can be beneficial to depict problem-solving process.
Approach:	They propose a method to retrieve exemplars with semantic and structural similarity using a graph kernel.
Outcome:	The proposed method is superior to state-of-the-art retrieval-based approaches on mathematics and logical reasoning tasks.

Copied to clipboard

Geling Liu, Yunzhi Tan, Ruichao Zhong, Yuanzhen Xie, Lingchen Zhao, Qian Wang, Bo Hu, Zang Li

Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale (2023.acl-long)

Challenge:	Existing text-to-SQL approaches have overlooked the critical aspect of system robustness.
Approach:	They propose a robust text-to-SQL solution that integrates with LLMs . their method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% .
Outcome:	The proposed solution achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks.

Copied to clipboard

Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff, Dan Roth

LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis (2023.findings-emnlp)

Challenge:	70% of attention heads and 20% of the feed forward networks can be removed with minimal decline in task performance.
Approach:	They propose to investigate whether in-context learning is not uniform across all components of a large language model.
Outcome:	The proposed model can remove 70% of attention heads and 20% of feed forward networks with minimal decline in task performance.

Copied to clipboard

Shih-Chieh Dai, Aiping Xiong, Lun-Wei Ku

Exploring In-Context Learning for Knowledge Grounded Dialog Generation (2023.findings-emnlp)

Challenge:	Recent research shows that large language models can replicate human-like behavior in various tasks.
Approach:	They propose a framework for human-LLM collaboration to conduct TA with in-context learning (ICL) they propose to use survey data to frame discussions with an LLM to generate a final codebook for TA.
Outcome:	The proposed framework outperforms crowd workers on text-annotation tasks and yields similar coding quality to that of human coders but reduces TA’s labor and time demands.

Copied to clipboard

Qinyu Chen, Wenhao Wu, Sujian Li

Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learning (2025.coling-main)

Challenge:	Existing knowledge grounded dialog generation models are prone to hallucination and produce factually inaccurate outputs.
Approach:	They propose a retrieval-based framework which leverages in-context learning and retrieval techniques to enhance LLMs on knowledge grounded dialog generation.
Outcome:	The proposed framework outperforms existing training-based models on a large-scale knowledge graph with 1M+ facts and is expected to perform knowledge-intensive tasks.

Copied to clipboard

Piyapath T. Spencer, Nanthipat Kongborrirak

Retrieval Enhanced Feedback via In-context Neural Error-book (2025.emnlp-main)

Challenge:	a new study examines the potential of large language models for documenting endangered languages . the model can be used to generate grammatical information for low-resource languages despite limitations .
Approach:	They examine the efficacy of LLMs in generating grammatical information for low-resource languages . they use bilingual dictionaries and parallel sentences of the unknown language as a case study .
Outcome:	The proposed model produces coherent grammatical rules and lexical entries using bilingual dictionaries and parallel sentences of the unknown language without building the model from scratch.

Copied to clipboard

Jongyeop Hyun, Bumsoo Kim

CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations (2025.acl-long)

Challenge:	Existing methods for learning from errors lack a structured framework for analyzing and mitigating errors, especially in Multimodal Large Language Models (MLLMs).
Approach:	They propose a teacher-student framework that systematically structures errors to deliver targeted feedback for multimodal reasoning.
Outcome:	The proposed framework improves inference efficiency, token usage, and scalability by building a query-based structure that prioritizes visual information, diagnoses failure points, and guides corrective actions.

Copied to clipboard

Vignesh Kothapalli, Hamed Firooz, Maziar Sanjabi

Using Natural Language Explanations to Improve Robustness of In-context Learning (2024.acl-long)

Challenge:	In-context learning and CoT are still poorly understood, but the precise mechanisms and architectural factors driving ICL and Co T are still unclear.
Approach:	They propose a framework and methodology to generate synthetic tokenized datasets and study chain-of-thought (CoT) in-context learning in language models.
Outcome:	The proposed framework and methodology allows fine grained control over the complexity of in-context examples by decoupling causal structure from underlying token processing functions.

Copied to clipboard

Xuanli He, Yuxiang Wu, Oana-Maria Camburu, Pasquale Minervini, Pontus Stenetorp

Robustness of Named-Entity Replacements for In-Context Learning (2023.findings-emnlp)

Challenge:	Recent studies show that large language models excel in many tasks via in-context learning (ICL). However, ICL struggles to execute complex tasks such as arithmetic, commonsense, and symbolic reasoning.
Approach:	They propose to augment ICL with natural language explanations (NLEs) to produce further NLEs on adversarial datasets.
Outcome:	The proposed approach yields more accurate results than zero-shot-ICL and using only human-generated NLEs on eight adversarial datasets.

Copied to clipboard

Saeed Goodarzi, Nikhil Kagita, Dennis Minn, Shufan Wang, Roberto Dessi, Shubham Toshniwal, Adina Williams, Jack Lanchantin, Koustuv Sinha

ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models (2024.acl-long)

Challenge:	Modern large language models perform in-context learning, where query- answer demonstrations are shown before the final query.
Approach:	They propose to use in-context learning to prompt queries before they are answered . they find that the choice of demonstrations can affect model performance .
Outcome:	The proposed model performance improves on named entity replacements across three reasoning tasks and two popular LLMs.

Copied to clipboard

Hojae Han, Jaejin Kim, Jaeseok Yoo, Youngwon Lee, Seung-won Hwang

Concept-aware Data Construction Improves In-context Learning of Language Models (2024.findings-acl)

Challenge:	Despite the critical role of software requirements, these criteria have not been studied actively in previous code generation works.
Approach:	They propose a framework that leverages in-context learning to organize and extrapolate unexpressed requirements from textual descriptions.
Outcome:	The proposed framework generates functional requirements from textual descriptions and extrapolates unexpressed requirements from them.

Copied to clipboard

Michal Štefánik, Marek Kadlčík, Petr Sojka

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering (2025.acl-long)

Challenge:	Recent work curating in-context learners assumes that ICL emerges from vast over-parametrization or the scale of multitask training.
Approach:	They propose a framework for constructing training scenarios that make it beneficial for the LM to learn to utilize the analogical reasoning concepts from demonstrations.
Outcome:	The proposed framework makes it beneficial for the LM to learn to utilize the analogical reasoning concepts from demonstrations and fares comparably to previous in-context learners trained in large-scale multitask learning requiring magnitudes of more training data.

Copied to clipboard

Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, Yunpu Ma

Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models (2024.findings-emnlp)

Challenge:	Multimodal Large Language Models (MLLMs) enhance visual tasks by integrating visual representations into large language models.
Approach:	They propose a method to re-balance modalities by steering visual representations . they propose LLaVA Steering, a platform that enables rapid customization of MLLMs a component-based architecture .
Outcome:	The proposed model re-balances the modalities of visual representations in large language models . the model requires 500 times fewer trainable parameters than LoRA while maintaining comparable performance .

Copied to clipboard

Kang He, Yinghan Long, Kaushik Roy

Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings (2025.findings-acl)

Challenge:	Prompt-based learning is susceptible to intrinsic bias present in pre-trained language models (LMs), leading to sub-optimal performance in prompt-based zero/few-shot settings.
Approach:	They propose a null-input prompting method to calibrate intrinsic bias encoded in pre-trained language models (LMs) they leverage a diverse set of auto-selected null meaning inputs generated from GPT-4 to probe intrinsic bias.
Outcome:	The proposed method significantly improves zero/few-shot learning performance of LMs for both in-context learning and prompt-based fine-tuning (on average 9% and 2%, respectively).

Copied to clipboard

Shujian Yang, Shiyao Cui, Chuanrui Hu, Haicheng Wang, Tianwei Zhang, Minlie Huang, Jialiang Lu, Han Qiu

DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph (2025.acl-long)

Challenge:	Recent studies show that character substitutions in toxic Chinese text can confuse state-of-the-art LLMs.
Approach:	They propose a taxonomy of 3 perturbation strategies and 8 specific approaches in Chinese text to assess if they can detect perturbed Chinese toxic contents.
Outcome:	The proposed model can detect perturbed Chinese text with 8 different approaches . the proposed model is compared with 9 other LLMs from the US and China .

Copied to clipboard

Jihyung Lee, Jin-Seop Lee, Jaehoon Lee, YunSeok Choi, Jee-Hyong Lee

ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Filler (2024.lrec-main)

Challenge:	Existing methods for Text-to-SQL show little improvement compared to random selections . Existing approaches rely on intrinsic capabilities of hyper-scaled LLMs, not useful demonstrations.
Approach:	They propose a novel approach to effectively retrieving demonstrations and generating SQL queries by linking a question and its database schema items.
Outcome:	The proposed method shows consistent improvements in performance and efficiency across hyper-scaled LLMs and small LLM.

Copied to clipboard

Paramita Mirza, Viju Sudhi, Soumya Ranjan Sahoo, Sinchana Ramakanth Bhat

AutoTrial: Prompting Language Models for Clinical Trial Design (2023.emnlp-main)

Challenge:	State-of-the-art intent classification and slot filling methods rely on data-intensive deep learning models . large language models exhibit remarkable zero-shot performance across various natural language tasks.
Approach:	They propose an approach framing IC and SF as language generation tasks for instruction-LLMs with a more efficient SF-prompting method.
Outcome:	The proposed approach outperforms state-of-the-art IC+SF method and in-context learning methods with GPT3.5 (175B).

Copied to clipboard

Zifeng Wang, Cao Xiao, Jimeng Sun

Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting (2024.findings-emnlp)

Challenge:	Generative large language models (LLMs) are a popular tool for creating coherent and human-like documents for clinical trials.
Approach:	They propose to generate clinical eligibility criteria using language models by a hybrid of discrete and neural prompting and scalable knowledge incorporation via in-context learning.
Outcome:	The proposed method generates high-quality criteria texts fluent and coherent with high accuracy against the GPT-3.5 baselines.

Copied to clipboard

Siyi Liu, Yang Li, Jiang Li, Shan Yang, Yunshi Lan

The Hidden Strength of Disagreement: Unraveling the Consensus-Diversity Tradeoff in Adaptive Multi-Agent Systems (2025.emnlp-main)

Challenge:	Existing methods for zero-shot Relation Extraction (RE) lack detailed, context-specific prompts for understanding various sentences and relations.
Approach:	They propose a framework that uses a three-stage diversity approach to prompt LLMs by generating multiple synthetic samples that encapsulate specific relations from scratch.
Outcome:	The proposed framework outperforms existing LLM-based zero-shot RE methods on benchmark datasets and shows that it produces high-quality synthetic data that enhances performance.

Copied to clipboard

Zengqing Wu, Takayuki Ito

In-Context Example Retrieval from Multi-Perspectives for Few-Shot Aspect-Based Sentiment Analysis (2024.lrec-main)

Challenge:	Conventional LLM-based MAS rely on explicit coordination, e.g., prompts or voting, risking premature homogenization.
Approach:	They propose to preserve partial diversity by combining in-context learning with explicit coordination to form consensus in dynamic environments.
Outcome:	The proposed model outperforms explicit consensus models on three scenarios showing that partial deviation from group norms boosts exploration, robustness, and performance.

Copied to clipboard

Qianlong Wang, Hongling Xu, Keyang Ding, Bin Liang, Ruifeng Xu

The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis (2024.emnlp-main)

Challenge:	Existing approaches to solve few-shot aspect-based sentiment analysis (ABSA) are suboptimal for this task because of in-context examples .
Approach:	They propose to retrieve in-context examples for few-shot aspect-based sentiment analysis . they construct positive and negative pairs from three perspectives and train the retriever .
Outcome:	The proposed retrieval framework outperforms baselines on four ABSA datasets.

Copied to clipboard

Yuxiang Zhou, Jiazheng Li, Yanzheng Xiang, Hanqi Yan, Lin Gui, Yulan He

Using In-Context Learning to Improve Dialogue Safety (2023.findings-emnlp)

Challenge:	In-context learning (ICL) is a capability that enables large language models to excel in proficiency through demonstration examples.
Approach:	They present a survey on the interpretation and analysis of in-context learning . they focus on theoretical and empirical perspectives on the concept .
Outcome:	The proposed model can perform tasks with minimal examples without re-training and has demonstrated proficiency across various tasks with a minimal set of task-oriented examples.

Copied to clipboard

Nicholas Meade, Spandana Gella, Devamanyu Hazarika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, Dilek Hakkani-Tur

RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information (2025.acl-long)

Challenge:	Recent work has highlighted safety issues with large neural-based conversational models.
Approach:	They propose a retrieval-based approach for reducing bias and toxicity in chatbot responses . they retrieve demonstrations of safe responses to similar dialogue contexts to generate a response .
Outcome:	The proposed method reduces bias and toxicity in three chatbot models . it can be used in compliment to existing dialogue safety approaches, such as RLHF.

Copied to clipboard

Zhiwei Liu, Kailai Yang, Qianqian Xie, Christine de Kock, Sophia Ananiadou, Eduard Hovy

Skills-in-Context: Unlocking Compositionality in Large Language Models (2024.findings-emnlp)

Challenge:	Current methods for cross-domain misinformation detection focus on in-domain tasks and do not incorporate significant sentiment and emotion features.
Approach:	They propose a retrieval augmented (RAG) LLM framework that incorporates affective information into retrieval databases.
Outcome:	The proposed framework improves on three misinformation benchmarks.

Copied to clipboard

Jiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song, Xiaoyang Wang, Dong Yu, Jianshu Chen

Structured Confidence–Guided Online Adaptation for LLM-based Multi-Label Classification (2026.findings-acl)

Challenge:	eliciting compositional generalization capabilities in large language models is challenging for advanced LLMs because they lack foundational skills and compositional examples in the same prompt context.
Approach:	They propose to use compositional generalization capabilities in large language models to elicit compositional skills in a prompt context.
Outcome:	The proposed structure enables LLMs to tackle more challenging problems with as few as two exemplars and unlocks their latent potential.

Copied to clipboard

Pengyu Xu, JingRen Hou, Liping Jing, Jian Yu

LLM as a metric critic for low resource relation identification (2024.findings-emnlp)

Challenge:	Large language models (LLMs) enable zero-shot and few-shot multi-label text classification . but most approaches perform static inference and degrade under streaming test data .
Approach:	They propose a structured confidence-guided online adaptation framework for LLM-based multi-label generation without parameter updates.
Outcome:	The proposed framework improves Micro-F1 and Macro-F1, with the largest gains on long-tail labels.

Copied to clipboard

Zhe Yang, Yi Huang, Yaqin Chen, Xiaoting Wu, Junlan Feng, Chao Deng

Skill-Based Few-Shot Selection for In-Context Learning (2023.emnlp-main)

Challenge:	Existing studies show that small language models (SLMs) overfit in low resource situations . however, the gap between pre-training and fine-tuning leads to performance decay .
Approach:	They propose to combine large language models and LLM for relation identification by co-evolution . they propose to use a masked language model prompt to generate a relation identification task .
Outcome:	The proposed model can handle low resource relation identification tasks with minimal overfitting . the proposed model provides essential background knowledge to assist training process .

Copied to clipboard

Shengnan An, Bo Zhou, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Weizhu Chen, Jian-Guang Lou

Ensemble-Instruct: Instruction Tuning Data Generation with a Heterogeneous Mixture of LMs (2023.findings-emnlp)

Challenge:	Existing methods based on pre-trained embeddings can be easily biased by surface features that are not important for the target task.
Approach:	They propose a skill-based few-shot selection method for in-context learning . it generates skill-specific descriptions for each test case and candidate example .
Outcome:	The proposed method significantly outperforms existing methods in five cross-domain semantic parsing datasets and six backbone models.

Copied to clipboard

Young-Suk Lee, Md Sultan, Yousef El-Kurdi, Tahira Naseem, Asim Munawar, Radu Florian, Salim Roukos, Ramón Astudillo

Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles (2024.findings-emnlp)

Challenge:	Empirical studies with different instruction-tuned LMs show that our proposed method yields higher-quality instruction tuning data than Self-Instruct.
Approach:	They propose to use in-context learning techniques to train strong conversational agents . they propose to categorize and simplify ICL templates to make prompt learning easier .
Outcome:	Empirical results show that the proposed method yields higher-quality instruction tuning data than Self-Instruct and improves performance of both vanilla and instruction-tuned LMs.

Copied to clipboard

Xiao Pu, Tianxing He, Xiaojun Wan

Exploring the Role of Reasoning Structures for Constructing Proofs in Multi-Step Natural Language Reasoning with Large Language Models (2024.emnlp-main)

Challenge:	Prompt compression reduces inference time and costs while maintaining informativeness for different usage scenarios.
Approach:	They propose a framework that adapts a smaller language model to compress prompts for a larger model on a new task without additional training.
Outcome:	The proposed framework outperforms two baseline models in four tasks . iteratively generates and selects effective compressed prompts as task-specific demonstrations .

Copied to clipboard

Zi’ou Zheng, Christopher Malon, Martin Min, Xiaodan Zhu

AppBench: Planning of Multiple APIs from Various APPs for Complex User Instruction (2024.emnlp-main)

Challenge:	Large language models (LLMs) are essential for performing complex multi-step reasoning tasks, such as multi-hop reasoning tasks.
Approach:	They propose to use large language models to derive structured intermediate proof steps to improve their performance by using examples.
Outcome:	The proposed models can derive correct proof steps with in-context learning.

Copied to clipboard

Hongru Wang, Rui Wang, Boyang Xue, Heming Xia, Jingtao Cao, Zeming Liu, Jeff Pan, Kam-Fai Wong

Structured Data in NLP

Challenge:	Existing state-of-the-art Large Language Models (LLMs) still cannot perform well in this situation even with the help of in-context learning and finetuning.
Approach:	They propose a benchmark to evaluate LLMs’ ability to plan and execute multiple APIs from various sources in order to complete the user’s task.
Outcome:	The proposed benchmarks show that the existing state-of-the-art LLMs still cannot perform well in this situation even with in-context learning and finetuning.

ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness? (2024.emnlp-main)

Copied to clipboard

Siddhant Waghjale, Vishruth Veerendranath, Zhiruo Wang, Daniel Fried

FSTs vs ICL: Generalisation in LLMs for an under-resourced language (2025.findings-emnlp)

Challenge:	Current methods for optimizing program efficiency improve performance measured by execution time, but they often come at the cost of severely decreasing the functional correctness.
Approach:	They propose a reproducible benchmark for evaluating program efficiency via two paradigms: natural language (NL) based code generation and history-based code editing.
Outcome:	The proposed approach improves performance while maintaining correctness while adding execution information.

Copied to clipboard

Ximena Gutierrez, Mikel Segura Elizalde, Victor Mijangos

Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing (2025.findings-emnlp)

Challenge:	LLMs have been widely adopted to tackle many traditional NLP tasks, but their effectiveness remains uncertain in scenarios where pre-trained models have limited prior knowledge of a language.
Approach:	They propose a rule-based method using a finite-state transducer and an in-context learning method that provides the model with string transduction examples.
Outcome:	The proposed method outperforms FSTs in zero-shot settings while ICL surpasses FLMs.

Copied to clipboard

Jongwoo Kim, SeongYeub Chu, Bryan Wong, Mun Yong Yi

Public Data Assisted Differentially Private In-Context Learning (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have strong reasoning and generalization abilities, but they struggle to reflect the histories of example learners within a single prompt during in-context learning.
Approach:	They propose a LLM-based option weighted knowledge tracing framework that encodes the interaction histories of example learners in context as textual categorical option weights.
Outcome:	The proposed framework outperforms existing LLM-based KT models in warm-start and few-shot settings.

Copied to clipboard

Seongho Joo, Hyukhun Koh, Kyomin Jung

Multi-Task Transfer Matters During Instruction-Tuning (2024.findings-acl)

Challenge:	In-context learning has shown remarkable performance across tasks without fine-tuning . however, recent studies have highlighted the risk of private data leakage through the prompt in ICL .
Approach:	They propose a private in-context learning algorithm that effectively balances privacy protection and model utility.
Outcome:	The proposed algorithm is robust against membership inference attacks and is robust to membership infertility attacks.

Copied to clipboard

David Mueller, Mark Dredze, Nicholas Andrews

Pioneering Reliable Assessment in Text-to-Image Knowledge Editing: Leveraging a Fine-Grained Dataset and an Innovative Criterion (2024.findings-emnlp)

Challenge:	Instruction-tuning improves a model’s ability to learn in-context, but the mechanisms that drive in-constext learning are poorly understood.
Approach:	They propose to train a model on hundreds of tasks to improve its ability to learn in-context.
Outcome:	The proposed methods improve model transfer and in-context generalization, suggesting catastrophic forgetting may impact in-constext learning.

Copied to clipboard

Hengrui Gu, Kaixiong Zhou, Yili Wang, Ruobing Wang, Xin Wang

CrystalICL: Enabling In-Context Learning for Crystal Generation (2025.emnlp-main)

Challenge:	Text-to-image models encode factual knowledge into their parameters, but they may become obsolete over time.
Approach:	They propose a framework for T2I knowledge editing that integrates paraphrase and multi-object test to enable more fine-grained assessment on knowledge generalization.
Outcome:	The proposed framework improves on existing models and improves their performance.

Copied to clipboard

Ruobing Wang, Qiaoyu Tan, Yili Wang, Ying Wang, Xin Wang

Distilling Many-Shot In-Context Learning into a Cheat Sheet (2025.findings-emnlp)

Challenge:	Existing methods for crystal generation are limited to zero-shot scenarios and are unable to benefit from few-shot situations.
Approach:	They propose a model designed for few-shot crystal generation that exploits in-context learning by capturing structure-property relationships from limited data.
Outcome:	The proposed model reduces complexity of modeling crystal symmetry in LLMs and exploits ICL by capturing structure-property relationships from limited data.

Copied to clipboard

Ukyo Honda, Soichiro Murakami, Peinan Zhang

Locally Differentially Private In-Context Learning (2024.lrec-main)

Challenge:	Recent advances in large language models (LLMs) enable effective in-context learning with many-shot examples, but at the cost of high computational demand due to longer input tokens.
Approach:	proposed cheat-sheet ICL distills information from many-shot ICL into a concise textual summary . experiment shows cheat- sheet ICL achieves comparable or better performance than many- shot ICL .
Outcome:	Experiments on reasoning tasks show that cheat-sheet ICL achieves comparable or better performance than many-shot ICL with far fewer tokens.

Copied to clipboard

Chunyan Zheng, Keke Sun, Wenhao Zhao, Haibo Zhou, Lixing Jiang, Shaoyang Song, Chunlai Zhou

SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questions (2024.findings-emnlp)

Challenge:	Large pretrained language models (LLMs) have shown surprising In-Context Learning ability.
Approach:	They propose a locally differentially private framework of in-context learning for LLMs that can be augmented with a private database for some specific task.
Outcome:	The proposed framework can predict labels without additional parameter modifications without input-label pairs .

Copied to clipboard

Shicheng Liu, Sina Semnani, Harold Triedman, Jialiang Xu, Isaac Zhao, Monica Lam

Multi-label Sequential Sentence Classification via Large Language Model (2024.findings-emnlp)

Challenge:	Large Language Models (LLMs) have led to significant improvements in the Knowledge Base Question Answering task.
Approach:	They introduce an expert-annotated KBQA dataset from Wikidata’s “Request a Query” forum with 320 decontextualized question-SPARQL pairs.
Outcome:	The SPINACH dataset outperforms baselines on the QALD-7, QADL-9 Plus and QAL-10 datasets by 31.0%, 27.0% and 10.0% in F1 respectively.

Copied to clipboard

Mengfei Lan, Lecheng Zheng, Shufan Ming, Halil Kilicoglu

Selective Demonstrations for Cross-domain Text-to-SQL (2023.findings-emnlp)

Challenge:	Existing approaches to sequential sentence classification are constrained by model size, sequence length, and single-label setting.
Approach:	They propose a large language model-based framework for both single- and multi-label SSC tasks that generate SSC labels through designed prompts.
Outcome:	The proposed framework enhances task understanding by incorporating demonstrations and a query to describe the prediction target.

Copied to clipboard

Shuaichen Chang, Eric Fosler-Lussier

Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches (2025.findings-emnlp)

Challenge:	Large language models with in-context learning have demonstrated impressive generalization capabilities in the cross-domain text-to-SQL task without the use of in-domain annotations.
Approach:	They propose a demonstration selection framework that utilizes both out-of-domain examples and synthetically generated in-domain demonstration examples to construct demonstrations.
Outcome:	The proposed framework outperforms baseline methods on two cross-domain text-to-SQL datasets with improvements of 1.1 and 11.8 points in execution accuracy.

Copied to clipboard

Israel Abebe Azime, Deborah D. Kanubala, Tejumade Afonja, Mario Fritz, Isabel Valera, Dietrich Klakow, Philipp Slusallek

DeCoVec: Building Decoding Space based Task Vector for Large Language Models via In-Context Learning (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are increasingly employed in high-stakes decision-making tasks such as loan approvals.
Approach:	They evaluate the performance and fairness of LLMs on serialized loan approval datasets from Ghana, Germany, and the United States.
Outcome:	The model’s zero-shot and in-context learning (ICL) capabilities are evaluated on loan approval datasets from Ghana, Germany, and the United States.

Copied to clipboard

Feiyang Li, Yile Wang

HyperLoRA: Efficient Cross-task Generalization via Constrained Low-Rank Adapters Generation (2024.findings-emnlp)

Challenge:	Existing approaches to steering large language models require fine-tuning or manipulation of internal states, limiting their flexibility and scalability.
Approach:	They propose a framework that constructs task vectors directly in the decoding space by leveraging in-context learning.
Outcome:	The proposed framework outperforms standard few-shot baselines on TruthfulQA, Math-500, and AQUA-RAT with gains up to +5.50 accuracy.

Copied to clipboard

Chuancheng Lv, Lei Li, Shitou Zhang, Gang Chen, Fanchao Qi, Ningyu Zhang, Hai-Tao Zheng

Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations (2023.emnlp-main)

Challenge:	Existing approaches to adapt pre-trained language models (PLMs) to emerging tasks are costly and inefficient.
Approach:	They propose a meta-network that generates task-specific weights without any optimization.
Outcome:	The proposed approach has flexible generalization ability and superior performance over hypenetworks.

Copied to clipboard

Wei-Lin Chen, Cheng-Kuang Wu, Yun-Nung Chen, Hsin-Hsi Chen

Tree of Problems: Improving structured problem solving with compositionality (2024.emnlp-main)

Challenge:	Large language models (LLMs) have shown striking ability to adapt to target tasks with a few input-output demonstrations.
Approach:	They propose a framework which bootstraps LMs’ intrinsic capabilities to perform zero-shot ICL.
Outcome:	The proposed framework outperforms baselines on 23 BIG-Bench Hard tasks on average accuracy and head-to-head comparison.

Copied to clipboard

Armel Zebaze, Benoît Sagot, Rachel Bawden

Topic Coverage-based Demonstration Retrieval for In-Context Learning (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable performance across multipletasks through in-context learning.
Approach:	They propose a Tree of Problems (ToP) that is a simpler version of Tree of Thoughts (toT) they propose 'in-context learning' is the ability of Large Language Models (LLMs) to perform a task with the help of a few demonstrations within their context.
Outcome:	The proposed approach outperforms ToT and GoT and performs better on complex reasoning tasks.

Copied to clipboard

Wonbin Kweon, SeongKu Kang, Runchu Tian, Pengcheng Jiang, Jiawei Han, Hwanjo Yu

TABGEN-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation (2025.findings-acl)

Challenge:	Prior methods to retrieve demonstrations based on embedding similarity or generation probability, resulting in irrelevant or redundant examples.
Approach:	They propose a topic coverage-based retrieval framework that selects demonstrations to comprehensively cover topic-level knowledge relevant to both the test input and the model.
Outcome:	The proposed framework covers all the necessary knowledge for the test input and the model.

Copied to clipboard

Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu

DocCGen: Document-based Controlled Code Generation (2024.emnlp-main)

Challenge:	Existing approaches to tabular data generation require fine-tuning, which is computationally expensive.
Approach:	They propose a new in-context learning framework to prompt a fixed LLM with in-constitut examples to enhance the in-text learning ability of LLMs for tabular data generation.
Outcome:	The proposed framework outperforms random selection strategies on five real-world tabular datasets and reduces error rate by 42.2% on fidelity metric.

Copied to clipboard

Sameer Pimparkhede, Mehant Kammakomati, Srikanth Tamilselvam, Prince Kumar, Ashok Kumar, Pushpak Bhattacharyya

G2S: A General-to-Specific Learning Framework for Temporal Knowledge Graph Forecasting with Large Language Models (2025.findings-acl)

Challenge:	Large language models (LLMs) produce state-of-the-art performance on natural language to code generation for resource-rich general-purpose languages like C++, Java, and Python.
Approach:	They propose a framework that breaks the NL-to-Code generation task into two steps . they use library documentation to detect the correct libraries and schema rules extracted from the documentation to constrain the decoding .
Outcome:	The proposed framework improves different sized language models across all six evaluation metrics, reducing syntactic and semantic errors in structured code.

Copied to clipboard

Long Bai, Zixuan Li, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng, Tat-Seng Chua

Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction (2025.findings-acl)

Challenge:	Recent studies have introduced Large Language Models (LLMs) for this task to enhance the models’ generalization abilities.
Approach:	They propose a General-to-Specific learning framework that disentangles the learning processes of two kinds of knowledge in a temporal temporal structure.
Outcome:	The proposed framework disentangles the learning processes of the above two kinds of knowledge and improves their generalization abilities.

Copied to clipboard

Guangyue Peng, Wei Li, Wen Luo, Houfeng Wang

Representation Learning

Challenge:	a novel method for encoding fine-grained error patterns improves performance on GEC.
Approach:	They propose a method for encoding grammatical errors from LLMs' internal states using a GER method.
Outcome:	The proposed method significantly boosts performance in ICL settings on multilingual GEC datasets.

Can Input Attributions Explain Inductive Reasoning in In-Context Learning? (2025.findings-acl)

Copied to clipboard

Mengyu Ye, Tatsuki Kuribayashi, Goro Kobayashi, Jun Suzuki

Will LLMs Replace the Encoder-Only Models in Temporal Relation Classification? (2024.emnlp-main)

Challenge:	interpreting the internal process of neural models has long been a challenge . despite rapid progress, there are still questions bridging the IA and MI eras .
Approach:	They propose to use input attribution methods to interpret in-context learning . they find that a certain simple IA method works best in large models .
Outcome:	The proposed method is the best for interpreting LLM-based ICL, but the larger the model, the harder it is to interpret it.

Copied to clipboard

Gabriel Roccabruna, Massimo Rizzoli, Giuseppe Riccardi

Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Properties (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) have shown promising performance in temporal reasoning tasks such as temporal question answering.
Approach:	They propose to use large language models to detect temporal relations between events with in-context learning and lightweight fine-tuning approaches to assess their performance.
Outcome:	The proposed models significantly underperform smaller encoder-only models based on RoBERTa in the Temporal Relation Classification task.

Copied to clipboard

Keunwoo Yu, Zheyuan Zhang, Fengyuan Hu, Shane Storks, Joyce Chai

Rule-Guided Extraction: A Hierarchical Rule Optimization Framework for Document-Level Event Argument Extraction (2025.findings-emnlp)

Challenge:	Emergent In-context Learning on Videos induces in-contact learning over video and text . eILeV-trained models outperform other off-the-shelf VLMs in few-shot video narration for novel, rare actions.
Approach:	They implement Emergent In-context Learning on Videos (EILeV) that induces in-contact learning over video and text by capturing key properties of pre-training data.
Outcome:	The proposed training paradigm outperforms off-the-shelf VLMs in few-shot video narration for novel, rare actions.

Copied to clipboard

Yue Zuo, Yuxiao Fei, Wanting Ning, Jiayi Huang, Yubo Feng, Lishuang Li

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding (2025.findings-acl)

Challenge:	Document-level event argument extraction (EAE) is a critical task in natural language processing.
Approach:	They propose an LLM-driven HiErarchical Rule Optimization framework that iteratively generates and selects optimal hierarchical rules.
Outcome:	The proposed framework outperforms few-shot supervised methods and outperformed state-of-the-art prompting baselines.

Copied to clipboard

Tuo Zhang, Tiantian Feng, Yibin Ni, Mengqin Cao, Ruying Liu, Kiana Avestimehr, Katharine Butler, Yanjun Weng, Mi Zhang, Shrikanth Narayanan, Salman Avestimehr

Bayesian Example Selection Improves In-Context Learning for Speech, Text and Visual Modalities (2024.emnlp-main)

Challenge:	a new study examines the performance of large vision-language models in understanding art . the Pun Rebus Art Dataset is a multimodal dataset for art understanding rooted in traditional Chinese culture .
Approach:	They propose a multimodal dataset for art understanding deeply rooted in traditional Chinese culture . they aim to facilitate the development of VLMs that can better understand culturally specific content .
Outcome:	The proposed dataset shows that state-of-the-art VLMs struggle with these tasks . the data will facilitate the development of VLM models that can better understand culturally specific content .

Copied to clipboard

Siyin Wang, Chao-Han Yang, Ji Wu, Chao Zhang

Evaluation of LLMs in Medical Text Summarization: The Role of Vocabulary Adaptation in High OOV Settings (2025.findings-acl)

Challenge:	Large language models (LLMs) can adapt to new tasks easily and efficiently in a training-free manner.
Approach:	They propose to use eBayesian in-context example selection method to extend the inference probability conditioned on in-constitut examples based on Bayes’ theorem to select in-strategy examples . Experimental results show the efficacy and robustness of their method on various models, tasks and modalities.
Outcome:	The proposed method is based on the eBayesian in-context example selection approach.

Copied to clipboard

Gunjan Balde, Soumyadeep Roy, Mainack Mondal, Niloy Ganguly

Summarization

Challenge:	Large Language Models (LLMs) have been successful in medical text summarization . however, they do not perform fine-grained evaluations under difficult settings .
Approach:	They show that large language models show a significant performance drop for data points with high concentration of out-of-vocabulary words or with high novelty.
Outcome:	The proposed model shows a significant performance drop for data points with high concentration of out-of-vocabulary words or with high novelty.

Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection (2025.emnlp-main)

Copied to clipboard

Jingbiao Mei, Jinghong Chen, Guangyu Yang, Weizhe Lin, Bill Byrne

Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation (2025.findings-emnlp)

Challenge:	Large Multimodal Models (LMMs) have shown promise in hateful meme detection, but they face limitations like sub-optimal performance and limited out-of-domain generalization capabilities.
Approach:	They propose a robust adaptation framework for hateful meme detection that enhances in-domain accuracy and cross-domain generalization while preserving the general vision-language capabilities of LMMs.
Outcome:	The proposed framework outperforms larger agentic systems in detecting hateful memes under adversarial attacks while maintaining the general vision-language capabilities of LMMs.

Copied to clipboard

Armel Randy Zebaze, Benoît Sagot, Rachel Bawden

TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation (2025.findings-emnlp)

Challenge:	generative large language models (LLMs) can perform in-context learning . machine translation (MT) has been shown to benefit from in-constitu examples .
Approach:	They propose a compositional translation paradigm that replaces naive few-shot MT with similarity-based demonstrations.
Outcome:	The proposed paradigm replaces naive few-shot MT with similarity-based demonstrations.

Copied to clipboard

Armel Randy Zebaze, Benoît Sagot, Rachel Bawden

Accurate and Data-Efficient Toxicity Prediction when Annotators Disagree (2024.emnlp-main)

Challenge:	In-context learning and similarity search have been shown to improve LLMs' performance in machine translation, but they lag behind when dealing with low-resource languages.
Approach:	They propose a method that uses an LLM to generate topic-specific target-side data in the LRL.
Outcome:	The proposed approach boosts LLM translation performance during in-context learning and fine-tuning.

Copied to clipboard

Harbani Jaggi, Kashyap Coimbatore Murali, Eve Fleisig, Erdem Biyik

MIBench: Evaluating Multimodal Large Language Models over Multiple Images (2024.emnlp-main)

Challenge:	Disagreement among annotators can reveal nuances in subjective tasks that lack a simple ground truth .
Approach:	They propose three approaches to predict annotator ratings on the toxicity of text . they integrate annotators' history, demographics, survey information into their models .
Outcome:	The proposed approach outperforms other methods in toxicity rating prediction.

Copied to clipboard

Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang, Chunfeng Yuan, Bing Li, Weiming Hu

Delta-KNN: Improving Demonstration Selection in In-Context Learning for Alzheimer’s Disease Detection (2025.acl-long)

Challenge:	Existing benchmarks and MLLMs focus on single-image input scenarios, leaving performance of ML models when handling multiple images underexplored.
Approach:	They propose a benchmark to evaluate fine-grained abilities of multimodal large language models in multi-image scenarios.
Outcome:	The proposed benchmark categorizes the multi-image abilities into three scenarios: MII, MKS and MIC.

Copied to clipboard

Chuyuan Li, Raymond Li, Thalia S. Field, Giuseppe Carenini

LASER: An LLM-based ASR Scoring and Evaluation Rubric (2025.emnlp-main)

Challenge:	Existing methods for in-context learning (ICL) perform poorly for AD diagnosis due to inherent complexity of task.
Approach:	They propose a demonstration selection strategy that leverages a delta score to assess the relative gains of each training example and a KNN-based retriever that dynamically selects optimal “representatives” for a given input.
Outcome:	The proposed model outperforms existing methods on two AD detection datasets and surpasses even supervised classifiers.

Copied to clipboard

Amruta Parulekar, Preethi Jyothi

LIST: Linearly Incremental SQL Translator for Single-Hop Reasoning, Generation and Verification (2025.findings-acl)

Challenge:	Standard ASR evaluation metrics like word error rate penalize morphological and syntactic nuances that do not significantly alter sentence semantics.
Approach:	They propose an LLM-based scoring rubric LASER that leverages state-of-the-art LLMs’ in-context learning abilities to learn from prompts with detailed examples.
Outcome:	The proposed scoring rubric combines state-of-the-art learning capabilities with a high correlation score with human annotations.

Copied to clipboard

Kaiyuan Guan, Ruoxin Li, Xudong Guo, Zhenning Huang, Xudong Weng, Hehuan Liu, Zheng Wei, Zang Li

Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives (2024.lrec-main)

Challenge:	Existing schema linking methods are not able to handle complex SQL queries.
Approach:	They propose a new algorithm that transforms SQL queries into grammatically verifiable sub-queries which are arranged sequentially to reflect single-hop reasoning steps.
Outcome:	The proposed algorithm achieves significant performance gains on the BIRD dataset and surpasses schema linking methods at comparable or better cost.

Copied to clipboard

Gustave Cortal

EoT: Evolution of Thoughts for Complex Reasoning Tasks (2025.findings-emnlp)

Challenge:	Sigmund Freud's interpretation of dreams has been central to understanding human consciousness for centuries.
Approach:	They propose to automate the annotation process by using a natural language framework . they evaluate the impact of model size, prediction order of characters, and consideration of proper names and character traits .
Outcome:	The proposed model performs better with a large language model while having 28 times fewer parameters.

Copied to clipboard

Qin Hua, Jiaqi Sun, Shiyou Qian, Dingyu Yang, Jian Cao, Guangtao Xue

Fair-CCD: Mitigating Bias in Large Language Models for Tabular Classification Through Context-Contrastive Decoding (2026.acl-long)

Challenge:	Existing studies focus on ensuring behavior fidelity, factuality or reliability in generated reasoning processes, but they neglect the simultaneous optimization of all three aspects for each thought.
Approach:	They propose a thought assessment method that is sensitive to knowledge and LLM behaviors . they use three scorers to evaluate each thought by considering domain context, semantic alignment, and behavior impact.
Outcome:	The proposed framework outperforms advanced approaches in knowledge-based complex tasks.

Copied to clipboard

Donghan Liu, Han Sun, Zhaohui Wang, Qin Li, Min Zhang

Submodular-based In-context Example Selection for LLMs-based Machine Translation (2024.lrec-main)

Challenge:	Prior work to mitigate fairness issues often employs subjective demonstration selection, leading to low controllability and limited stability across different models and tasks.
Approach:	They propose to use in-context learning to insert social biases into large language models to create a structured and controllable representation of the relationship between sensitive attributes and predicted labels.
Outcome:	Extensive experiments show that Fair-CCD consistently improves fairness metrics without degrading task accuracy.

Copied to clipboard

Baijun Ji, Xiangyu Duan, Zhenyu Qiu, Tong Zhang, Junhui Li, Hao Yang, Min Zhang

Soft Head Selection for Injecting ICL-Derived Task Embeddings (2026.findings-acl)

Challenge:	Prior studies have focused on the role of well-chosen examples in in-context learning .
Approach:	They propose to use multiple translational factors for in-context example selection by using monotone submodular function maximization.
Outcome:	The proposed approach outperforms random selection and robust single-factor baselines across various NLP tasks.

Copied to clipboard

Jungwon Park, Jimyeong Kim, Changin Choi, Wonjong Rhee

Correlation-Aware Example Selection for In-Context Learning with Nonsymmetric Determinantal Point Processes (2025.emnlp-main)

Challenge:	Large language models (LLMs) are commonly adapted to downstream tasks using parameter-efficient fine-tuning (PEFT) or in-context learning (ICL).
Approach:	They propose a gradient-based method that derives task-specific embeddings from activations using few-shot prompts and injects them during inference.
Outcome:	The proposed method outperforms existing methods on open-ended generation, reasoning, and natural language understanding tasks while using fewer trainable parameters.

Copied to clipboard

Qiunan Du, Zhiliang Tian, Zhen Huang, Kailun Bian, Tianlun Liu, Zhaoning Zhang, Xinwang Liu, Feng Liu, Dongsheng Li

SSA: Improving Performance With a Better Scoring Function (2026.acl-long)

Challenge:	Existing studies on in-context learning (ICL) focus on the selection of individual examples and ignore correlations among examples.
Approach:	They propose a method to capture positive and negative correlations using the determinantal point process . they optimize the method via kernel decomposition-based MLE to fit a constructed pseudo-labeled dataset .
Outcome:	The proposed method outperforms baselines in ICL example selection.

Copied to clipboard

Omar Naim, Swarnadeep Bhar, Jerome Bolte, Nicholas Asher

Pretraining Context Compressor for Large Language Models with Embedding-Based Memory (2025.acl-long)

Challenge:	Despite the success of in-context learning, recent studies have identified systematic limitations in its generalization behavior.
Approach:	They propose a new attention scoring function that mitigates failures in transformer models . they use Scaled Signed Averaging to train the scoring function instead of Softmax .
Outcome:	The proposed scoring function outperforms transformer models with Softmax on NLP benchmarks and linguistic probing tasks.

Copied to clipboard

Yuhong Dai, Jianxun Lian, Yitian Huang, Wei Zhang, Mingyang Zhou, Mingqi Wu, Xing Xie, Hao Liao

Towards Robust In-Context Learning for Machine Translation with Large Language Models (2024.lrec-main)

Challenge:	Efficient processing of long contexts in large language models is essential for real-world applications such as retrieval-augmented generation and in-context learning.
Approach:	They propose a decoupled compressor-LLM framework that preserves contextual information within condensed embedding representations.
Outcome:	The proposed framework outperforms baseline models in three domains and across eight datasets while adapting to different downstream LLMs.

Copied to clipboard

Shaolin Zhu, Menglong Cui, Deyi Xiong

Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study (2024.lrec-main)

Challenge:	Experimental results demonstrate the effectiveness of our method, particularly in domain adaptation.
Approach:	They propose a method to retrieve translation pairs as demonstrations from an additional datastore to guide translation without updating the LLMs.
Outcome:	The proposed method reduces noise and improves translation performance in domain adaptation.

Copied to clipboard

Yaxin Fan, Feng Jiang, Peifeng Li, Haizhou Li

It’s All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs (2025.emnlp-main)

Challenge:	Large language models have shown remarkable capability in many downstream tasks, yet their ability to understand discourse structures of dialogues remains less explored.
Approach:	They aim to systematically inspect ChatGPT’s performance in two discourse analysis tasks: topic segmentation and discourse parsing.
Outcome:	The proposed model can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures.

Copied to clipboard

Yue Li, Zhixue Zhao, Carolina Scarton

Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning (2026.findings-acl)

Challenge:	Low-resource languages, especially those written in rare scripts, remain unsupported by large language models due to lack of training data.
Approach:	They evaluate 20 under-represented languages across three state-of-the-art multilingual LLMs and compare their methods to parameter-efficient fine-tuning.
Outcome:	The proposed methods compare with parameter-efficient fine-tuning (PEFT) on low-resource languages.

Copied to clipboard

Tiehua Mei, Minxuan Lv, Leiyu Pan, Zhenpeng Su, Hongru Hou, Hengrui Chen, Ao Xu, Deqing Yang

Learning on Imbalanced Noisy Data via Debiased Sample Selection and LLM-Driven Annotation (2026.findings-acl)

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance.
Approach:	They propose a method that reweights rewards by a factor approximately proportional to Evidence Gain and assigns higher weights to high-quality traces without requiring costly computation.
Outcome:	Experiments on mathematical reasoning benchmarks show that Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally.

Copied to clipboard

Bo Yuan, Yulin Chen, Yin Zhang

Robustness in NLP

Challenge:	Existing approaches to learning with noisy labels are prone to selection bias and training bias . obtaining large-scale high-quality datasets is expensive and time-consuming in practical scenarios .
Approach:	They propose an imbalanced learning with noisy labels task to let model learn from noisy labels . they first conduct debiased sample selection to better separate clean samples from noisy samples . then they feed selected clean samples to active annotator large language models for re-annotating noisy samples.
Outcome:	The proposed method is superior to existing methods on synthetic and real-world datasets.

Beyond Task-Level Context: Class-Conditional Context Vectors for Implicit In-Context Learning (2026.findings-acl)

Copied to clipboard

Jianxin Zhang, Yilu Hu, Teng Liu, Pei Guo, Juntao Li

BCL: Bayesian In-Context Learning Framework for Information Extraction (2026.findings-acl)

Challenge:	Existing approaches aggregate demonstrations from all classes into a shared, task-level context vector, capturing global task information but without explicitly preserving fine-grained, class-conditional semantic distinctions.
Approach:	They propose a class-conditional context vector extension to implicit in-context learning that explicitly models class-specific contextual information by constructing separate context vectors for each class.
Outcome:	The proposed extension outperforms task-level context vector baselines and achieves higher average accuracy than conventional few-shot learning on most models.

Copied to clipboard

Haoliang Liu, Chengkun Cai, Xu Zhao, Han Zhu, Shizhou Huang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, Zhang Huaping, Lei Li

Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning (2025.acl-long)

Challenge:	Existing information extraction (IE) tasks rely on in-context learning with large language models.
Approach:	They propose a Bayesian-based in-context learning framework that refines label representations across IE tasks using particle filtering and Bayes updates.
Outcome:	The proposed framework improves performance over existing methods (up to 30%) it underperforms one-shot prompting by a substantial margin on NER tasks and CodeIE fails on RE tasks with near-zero micro-F1.

Copied to clipboard

Chengwei Qin, Wenhan Xia, Fangkai Jiao, Chen Chen, Yuchen Hu, Bosheng Ding, Ruirui Chen, Shafiq Joty

Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval (2025.emnlp-main)

Challenge:	Existing methods to train student models on the generated outputs of teacher models are not efficient for ICL.
Approach:	They propose to align the output of smaller (student) models with that of larger (teacher) models by incorporating a ranking loss and aligning the token-level output distribution.
Outcome:	The proposed model outperforms baseline models on a variety of tasks involving language understanding, reasoning, and coding.

Copied to clipboard

Subhendu Khatuya, Shashwat Naidu, Pawan Goyal, Niloy Ganguly

MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses (2025.emnlp-main)

Challenge:	Numerical reasoning remains a challenging area for large language models (LLMs).
Approach:	They propose a two-step framework to enhance LLM's capabilities in financial numerical reasoning by using a generative retriever and context-aware program of thought prompting.
Outcome:	The proposed model surpasses previous benchmarks with execution accuracy improvements of 5.98% and 4.05%, respectively.

Copied to clipboard

Tong Chen, Zimu Wang, Yiyi Miao, Haoran Luo, Sun Yuanfei, Wei Wang, Zhengyong Jiang, Procheta Sen, Jionglong Su

Controlled Generation for Private Synthetic Text (2025.emnlp-main)

Challenge:	Existing medical fact-checking datasets focus on human-generated content, leaving the verification of content generated by large language models (LLMs) relatively unexplored.
Approach:	They propose to use Chinese medical fact-checking datasets to verify LLM-generated medical content by combining in-context learning and fine-tuning.
Outcome:	The first evidence-based Chinese medical fact-checking dataset of LLM-generated medical content consists of 1,321 questions and 7,409 claims .

Copied to clipboard

Zihao Zhao, Anjalie Field

WojoodRelations: Arabic Relation Extraction Corpus and Modeling (2025.emnlp-main)

Challenge:	Text anonymization is essential for developing and deploying AI in high stakes domains . tools for redacting directly identifying content are unlikely to guarantee 100% recall .
Approach:	They propose a method for privacy-preserving synthetic text generation that leverages HIPS theory and de-identification principles.
Outcome:	The proposed method achieves a strong balance between privacy protection and utility on legal and clinical datasets.

Copied to clipboard

Alaa Aljabari, Mohammed Khalilia, Mustafa Jarrar

Confidence-Aware Ranker Ensembles for Robust In-Context Knowledge Editing (2026.findings-acl)

Challenge:	Existing work on Arabic RE remains limited due to the language’s rich morphology and syntactic complexity, and the lack of large, high-quality datasets.
Approach:	They propose to use WojoodRelations to extract relation relationships from Arabic textual data using relation-aware templates and GPT-Joint to perform relation-based retrieval.
Outcome:	The proposed method achieves a Cohen’s of 0.92, indicating high reliability, and supervised models achieve 92.89% F1 for RE, while LLMs obtain 72.73% F1 .

Copied to clipboard

Tejal Nair, Mahmud Wasif Nafee, Maiqi Jiang, Ashley Gao, Haipeng Chen, Yanfu Zhang

Incomplete In-context Learning (2026.acl-long)

Challenge:	Large language models excel at factual recall, but can propagate stale or incorrect knowledge.
Approach:	They propose a feature-weighted ensemble for in-context knowledge editing that calibrates three heterogeneous rankers and extracts simple confidence features from each ranker.
Outcome:	The proposed method achieves 88.33% Edit-Success Rate over the best single retriever . it improves edit accuracy without touching model weights and approaches oracle upper bound (91%).

Copied to clipboard

Wenqiang Wang, Wen Yujia, Yan Xiao, Zhifeng Chen, Yangshijie Zhang, Peng Chen, Mingbo Yang, Xiaochun Cao

Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective (2026.acl-long)

Challenge:	Existing in-context learning assumes the retrieval dataset contains demonstrations for all output label spaces.
Approach:	They propose a framework with train-free and train-based variants to address IICL . they propose to integrate a dataset with labeled demonstrations for each output space .
Outcome:	The proposed framework outperforms existing methods under incomplete retrieval datasets and even outperformed ICL with complete labels.

Copied to clipboard

Bishwamittra Ghosh, Soumi Das, Till Speicher, Qinyuan Wu, Mohammad Aflah Khan, Deepak Garg, Krishna P. Gummadi, Evimaria Terzi

Many-Shot Scaling of In-Context Learning with Self-Generated Demonstrations (2026.findings-acl)

Challenge:	Prior studies comparing FT and ICL have yielded mixed and inconclusive results due to inconsistent experimental setups.
Approach:	They propose a formal language learning task with precise language boundaries, controlled string sampling, and no data contamination to enable a rigorous comparison.
Outcome:	The proposed task offers precise language boundaries, controlled string sampling, and no data contamination.

Copied to clipboard

Zhengyao Gu, Henry Peng Zou, Yankai Chen, Aiwei Liu, Weizhi Zhang, Philip S. Yu

Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text (2026.acl-long)

Challenge:	In-context learning methods that use self-generated annotations do not scale to many-shot scenarios.
Approach:	They propose a framework analogous to semi-supervised learning that uses self-generated annotations instead of ground truth labels.
Outcome:	The proposed framework outperforms ground truth ICL under zero-shot, few-shot and many-shot settings.

Copied to clipboard

Amr Mohamed, Yang Zhang, Michalis Vazirgiannis, Guokan Shang

Rose-SQL: Role-State Evolution Guided Structured Reasoning for Multi-Turn Text-to-SQL (2026.acl-long)

Challenge:	Code-switching (CSW) is widespread in multilingual communities and increasingly prevalent in online content.
Approach:	They propose a pipeline for producing linguistically grounded CSW variants of established benchmarks across five typologically diverse languages.
Outcome:	The proposed model sets show that inserting non-English tokens into English reduces accuracy on comprehension and reasoning benchmarks, whereas embedding English into non- English contexts often improves it.

Copied to clipboard

Le Zhou, Feng Yao, Fengcai Qiao, Bo Xu, Fangyuan Wang, Boyan Xu

Combining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction (2026.acl-long)

Challenge:	Existing approaches to multi-turn Text-to-SQL tasks rely on unstable APIs or expensive fine-tuning.
Approach:	They propose a training-free framework that leverages small-scale LRMs through in-context learning to enable accurate context-dependent parsing.
Outcome:	The proposed framework outperforms in-context learning baselines at the 4B scale and surpasses state-of-the-art models at the 8B and 14B scales.

Copied to clipboard

Vipul Kumar Rathore, Malik Hammad Faisal, Parag Singla, null Mausam

Wenqiang Wang, Peng Chen, Yan Xiao, Yangshijie Zhang, Xiaoyue Lu, Jianjie Huang, Xiaochun Cao

Challenge:	Existing Distantly Supervised Relation Extraction models rely on task-specific training, but their integration with in-context learning (ICL) using large language models (LLMs) remains underexplored.
Approach:	They propose a framework for distantly supervised relation extraction that uses a trained DSRE model to identify the top-k candidate relations for a given test sentence and a dynamic exemplar retrieval strategy that extracts reliable, sentence-level exemplars from training data.
Outcome:	The proposed framework achieves 20 F1 points gains in English and 17 F1 point gains on Indic languages over previous models and naive prompting baselines.

Copied to clipboard